Option (D) is correct.
The plan is to write sinB and cosB in terms of A, then use sin2B+cos2B=1 to remove B.
Step 1 — sinB in terms of A.
sinA=msinB⟹sinB=msinA
Step 2 — cosB in terms of A.
cosAsinA=ncosBsinB=n⋅mcosBsinA
sinA=0 (as A is acute), so cancel it:
cosA1=mcosBn⟹cosB=mncosA
Step 3 — use the identity for B.
m2sin2A+m2n2cos2A=1⟹sin2A+n2cos2A=m2
Step 4 — write sin2A=1−cos2A and solve.
1−cos2A+n2cos2A=m2
cos2A(n2−1)=m2−1
cos2A=n2−1m2−1
Check with numbers. Take A=60∘, B=30∘: tanA=3=3tanB, so n=3; sinA=23=3sinB, so m=3. Then 9−13−1=41=cos260∘. ✓
Why (A) is the trap: it is the correct fraction turned upside down — the value of cos2A1=sec2A.