The leader of the left-wing Economic Freedom Fighters MPs said the president has made the country “a joke”.
The speech is seen as a watershed moment for Mr Zuma, who is facing a court case over the use of $23m (£15m) of state money to upgrade his home.
The opposition has rejected Mr Zuma’s offer to repay some of the money.
Before being ejected from parliament in Cape Town, the EFF MPs chanted “Zupta must fall”, referring to the president’s alleged links to the influential Gupta family.
“Zuma is no longer a president that deserves respect from anyone,” EFF leader Julius Malema said.
Earlier, hundreds of opposition supporters took to the streets in Cape Town to protest over a range of issues, including Mr Zuma’s handling of the economy.
But supporters of the governing African National Congress (ANC) demonstrated to show their loyalty to Mr Zuma.
At the scene: Karen Allen, BBC News
This was in contrast to the glamorous ball gowns and national costumes worn by some of the guests on the red carpet.
Singing songs of land ownership and insulting the president with crude songs referring to the Nkandla scandal as they entered parliament, they were the ones everyone here was watching, even if some ANC figures studiously tried to ignore EFF leader Julius Malema and his noisy crowd.
The EFF promised to obey the rules of parliament but challenged Mr Zuma’s speech with interruptions.
Police used stun grenades after clashes broke out between rival protesters in the city’s central business district.
Riot police also set up barricades to keep protesters away from parliament.
When he was eventually able to speak, Mr Zuma said there was work to be done to turn the economy around and cut wastage.
“We will have to go through a difficult time for a while,” he said.
Mr Zuma is under intense pressure to deliver a plan to improve the country’s struggling economy, the BBC’s Milton Nkosi in South Africa says.
The economy is expected to grow less than 1% this year. Other problems he faces include unemployment at around 25%, poverty and a resurgence of public racial animosity.
To cut costs, Mr Zuma said South Africa would reconsider maintaining two capitals – the administrative capital in Pretoria and the legislative capital in Cape Town.
The president said he was working to attract foreign investment and mentioned the risk of the country being downgraded by ratings agencies.
“If that happens, it will become more expensive for us to borrow money from abroad to finance our programmes,” Mr Zuma said.
Another measure includes the creation of a state-owned pharmaceutical company that would compete with local firms in supplying medicines to public hospitals.
Mr Zuma also announced that South Africa would procure nuclear energy “on a pace and scale we can afford” to address chronic electricity shortages.
The president also addressed the resurgent racial tension in the country following the outrage caused by a Facebook post in which a white woman called black beachgoers “monkeys”.
“There is a need to confront the demon of racism,” Mr Zuma said.
He added that South Africa’s Human Rights Day on 21 March will be commemorated as a national day against racism: “It will be used to lay the foundation for a long-term programme of building a non-racial society.”
Opposition parties have taken the issue of Mr Zuma’s home upgrade to the Constitutional Court, hoping it may open the way for impeachment proceedings against him.
In court papers, Mr Zuma said he was prepared to repay money for non-security features, including the building of a swimming pool and an amphitheatre at his home in rural Nkandla in KwaZulu-Natal province.
However, opposition parties have pressed on with the case, asking judges to rule that Mr Zuma had violated the constitution and his oath of office by failing to repay the money when an anti-corruption watchdog first ruled in 2014 that he had “unduly benefited” from the renovations.