Preheat the oven to 350 degrees and prepare a 5x9 loaf pan.
Make the cream cheese filling first. Using and electric mixer, mix the cream cheese, sugar, egg yolk and vanilla together until it is smooth and creamy. Set it aside.
In a separate small mixing bowl, whisk together the flour, xanthan gum, sugar, baking soda, cinnamon, salt and nutmeg.
In a separate large mixing bowl, beat the egg using a fork.
Add the milk, melted butter and vanilla to the beaten egg and stir it all together.
Using a spatula (*DO NOT use electric mixer) add the pumpkin puree and flour mixture and mix just until combined. Do not over mix.
Pour 1/3 third of the pumpkin batter into the prepared loaf pan.
Then add a layer of cream cheese filling. Use a spatula to even out the filling later. Add another layer of pumpkin batter, then the rest of the filling, followed by the rest of the batter. (The top layer of pumpkin might be thin, that's ok). Smooth the top with a spatula.
Use a knife to cut through the batter horizontally to create a swirl effect.
Bake about 1 hour and 10 minutes until a toothpick inserted in the center comes out clean. Turn off the oven leaving the bread inside and crack the oven door open to allow for a slow temperature adjustment. Leave it in the oven for about 5 minutes more with the door cracked open.
Let the loaf cool in the pan for 30 minutes, then use a knife to loosen the edges of the bread and remove it from the pan to cool on a wire rack for about an hour.
Slice, serve and enjoy.