Rewrite the top and the bottom. By cosC−cosD=−2sin2C+Dsin2C−D and 1−cos2θ=2sin2θ:
cos5x−cos3x=−2sin4xsinx,1−cos4x=2sin22x
So
1−cos4xcos5x−cos3x=−sin22xsin4xsinx
Bring in the standard limits. Write each sine as (its standard ratio) × (its angle):
−sin22xsin4xsinx=−(2xsin2x)2⋅4x24xsin4x⋅xsinx⋅4x2=−(2xsin2x)24xsin4x⋅xsinx
Each ratio tends to 1, so the limit is −121×1=−1.