Fiscal discipline is as much needed as monetary discipline. Many countries have attempted to counter the deficit bias by adopting fiscal rules that typically set a limit to their annual budget deficits. The record is not satisfactory; rules are either too lax or too tight and then ignored. This article suggests that the solution is to adopt the approach followed by inflation targeting central banks, with great success. Independent and accountable Fiscal Policy Committees, given the task of achieving debt targets and the authority to decide - or recommend - annual deficits, will be free from the deficit bias. This will allow them to exercise discretion in the short run while delivering debt sustainability in the long run.