Keep that oven at 350°F. Grease and line an 8-inch round cake pan with parchment.
In a medium bowl, whisk together the almond flour, GF flour, baking powder, baking soda, and salt.
In another bowl, massage the zest into the sugar until it becomes fragrant. Then whisk sugar, olive oil, eggs, and vanilla until smooth. Stir in buttermilk.
Combine wet and dry ingredients. Stir until just mixed.
Pour into prepared pan, smooth the top, and bake for 35–40 minutes, or until golden and a tester comes out clean.
Cool in pan for 10 minutes, then move to a wire rack to cool completely.