If α + β + γ = 2π, then the system of equations
x + (cos γ)y + (cos β)z = 0
(cos γ)x + y + (cos α)z = 0
(cos β)x + (cos α)y + z = 0
has :

Solution

Maths sir · 4 steps

A homogeneous system always has the zero solution, so just check whether the determinant is zero.

1. Homogeneous system. Every right-hand side is 0, so x = y = z = 0 always works and the system can never have no solution. If D ≠ 0 that is the only solution. If D = 0 there are infinitely many.

   D = |1, cos γ, cos β; cos γ, 1, cos α; cos β, cos α, 1| = 1 - cos² α - cos² β - cos² γ + 2 cos α cos β cos γ

2. Use the condition. Since α + β + γ = 2π, we have cos(α+β) = cos(2π - γ) = cos γ. Also cos² α + cos² β = 1 + cos(α+β)cos(α-β) = 1 + cos γ cos(α-β).
3. Simplify D. Write 2 cos α cos β = cos(α+β) + cos(α-β) = cos γ + cos(α-β). Then D = -cos γ cos(α-β) - cos² γ + cos γ(cos γ + cos(α-β)) = 0.
4. So. The system is homogeneous and D = 0, so it has infinitely many solutions. That is option (b).

The idea to keep. For a homogeneous system, check only D. If D ≠ 0, only the trivial solution exists. If D = 0, there are infinitely many solutions. No solution is never possible.

Answer from the NTA final answer key. Solution by PrepWiser.