First , you should have changed the oil @ the 10 hour mark (per manual) if I'm not mistaken. Maybe you already have...

If you will do a google search on something like "F115 making oil" you will see that the problem has been inherent with the 4 stroke Yamaha's , especially the F115. The problem is the rings aren't seating yet. Yamaha tech's will tell you to run the engine much harder and get those rpms up. It will seat the rings properly and stop the gas from reaching the crankcase. I have also read that you should only fill with oil up to the half-way full mark on the dipstick to allow for some increase of liquid in the case.

I did a ton of research on it after I bought my current rig and found out what was causing it. I too noticed the level on the dipstick seemed to be rising.
My engine had 38 hours on it when I purchased my boat and the owner broke the motor in by trolling for crappie..LOL

The harder I run the motor the better it seems to perform so Don't baby it !!!

They are great little engines..hope you enjoy yours