Set the oven to 350°F.

Arrange the bread cubes evenly on an 18x13-inch rimmed baking sheet, then bake until they’re almost dry, about 10 minutes, tossing them halfway through. Take them out of the oven and allow them to cool.

Lower the oven temperature to 300°F. In a large mixing bowl, whisk together the brown sugar, granulated sugar, cinnamon, and nutmeg, breaking up any brown sugar clumps with your fingertips as needed.

Whisk in the eggs and egg yolks until fully incorporated. Next, mix in the milk, heavy cream, and vanilla extract until everything is smoothly blended.

Add the bread cubes to a large mixing bowl, then pour the milk mixture over them. Gently toss with a rubber spatula to coat the bread evenly. Let it sit for 15-30 minutes, tossing occasionally, until the milk mixture is fully absorbed.

Brush melted butter over the bottom of a 12x8-inch casserole dish. Pour in the bread mixture, spreading it out evenly. Bake in a preheated 300°F oven for 50-65 minutes, or until the center is set.

Allow it to cool for 10 minutes, then cut into squares. Serve with salted caramel or vanilla sauce and, if desired, a scoop of ice cream.