Preheat oven to 350°F. Grease a 2-quart baking dish with non-stick spray or butter.
In large mixing bowl, add drained sweet potato yams or fresh baked, peeled and slightly cooled sweet potatoes. Blend with an hand mixer until smooth.
To the bowl, add brown sugar, evaporated milk, melted butter, vanilla extract, cinnamon, nutmeg (if using), and eggs. Mix until smooth and well blended.
Pour the sweet potato mixture into the prepared baking dish and spread it evenly.
Using a cookie scoop or two spoons, add the marshmallow fluff in 1 tablespoon dollops all over the sweet potato mixture. Spooning from below, cover the dollops with the sweet potato mixture.
In a small bowl, combine chopped pecans, brown sugar, flour, and melted butter. Mix until the ingredients are well combined.
Sprinkle the pecan mixture evenly over the sweet potato layer.
Bake 40-50 minutes or until the topping is golden brown and the casserole is heated through.
Allow it to cool slightly before serving.