Combine pumpkin purée, evaporated milk, pudding mix, and pumpkin pie spice in a blender or bowl, and mix for about 2 minutes. Set aside to let pudding rest, about 5 minutes.
Blend or crush cookies until nicely crumbled.
Once pudding has set, spoon 2-3 tbs into each bowl to cover the bottom third. The amount may vary depending on the container's size.
Fill the next third of the bowl with the cookie crumbles.
Spoon in enough pudding to fill the containers to the top.
Pipe chocolate frosting onto marshmallows, then place marshmallows on the pudding.