Transparency for logos and graphics

I have a north arrow as a png and a logo as a png, both with transparent backgrounds.
I have added a data frame and added these as data. These data frames are on top of my main map data frame, and in ArcMap, the background of the png images stay transparent. All looks good in Arcmap.

When I go to export the map, I get a white background in those data frames. I export to pdf, but I've also tried to png.

Is there some setting I'm missing when I export?