Adjust the oven rack to the middle position and preheat the oven to 325 degrees.
Spread the almonds on a baking sheet and toast in the oven until lightly browned, about 10 to 15 minutes. Shake the pan halfway through to ensure that the nuts toast evenly and to check that the nuts are not browning too quickly. Cool, then coarsely chop half of the almonds.
In a small saucepan, clarify the butter by heating it over medium-high heat until it boils with large, loud, rapidly bursting bubbles. Turn the heat down to medium and continue cooking about 5 to 7 more minutes, without letting it brown, until the butter becomes foamy and the bubbles are fewer and quieter. Remove from heat and transfer to a bowl to cool. Skim the foam off the top and pour or spoon out 2/3 cup of the clarified butter, leaving the milk solids at the bottom of the bowl. Refrigerate any leftover butter for another use.
Turn the oven up to 350 degrees.
In the bowl of a food processor fitted with the steel blade, combine the whole almonds with half of the granulated sugar and process until it's the consistency of fine meal. Add the remaining sugars, flours, and salt, and pulse to incorporate. Add the egg yolks and orange flower water, and pulse just to combine.
Add the 2/3 cup of clarified butter and pulse on and off a few times until the mixture is just moistened, but does not come together. Transfer the dough to a mixing bowl and toss in the remaining chopped almonds.
Line a baking sheet with parchment or a silicone liner. Pick up a handful of dough, squeeze it together lightly in your fist, and crumble it, in large chunks, onto the baking sheet. Continue a handful at a time, crumbling the remaining dough.
Bake for 20 minutes, rotate the baking sheet and bake for another 20 minutes until lightly brown and firm to the touch. Do not overbake this, or else you will lose the delicate taste of the nuts and orange flower water. Cool completely and break up into uneven chunks.