Option (B) is correct.
Explanation: For an ideal gas PV=μRT, so at a fixed pressure the density ρ=RTPM is inversely proportional to the absolute temperature:
ρoutρflue=TflueTout=360300=0.833
The flue gas is 16.7% less dense. The student's coefficient does not apply here: for an ideal gas αV=1/T, which is 3.66×10−3 K−1 only at 273 K, and it changes as the gas warms. Even the value at 300 K, 1/300 K−1, gives the volume increase, 60/300=20%; a 20% larger volume means a density smaller by a factor 1/1.2=0.833, a 16.7% decrease.
(A) is wrong: it uses 3.66×10−3×60=0.22, the coefficient for 0 ∘C applied to air starting at 27 ∘C, and treats the volume increase as a density decrease.
(C) is wrong: 20% is the increase in volume; the density falls by 1−1/1.2=16.7%.
(D) is wrong: it uses Celsius temperatures, 1−27/87; gas laws need absolute temperatures.