BBC reports that opposition supporters accused Mr Sargsyan, who was made prime minister last week after serving 10 years as president, of clinging to power.
"The street movement is against my tenure. I am fulfilling your demand," Mr Sargsyan said in a statement.
Former prime minister Karen Karapetyan has taken over as acting PM. President Armen Sarkissian accepted Mr Sargsyan's and the government's resignation.
Mr Sargsyan's announcement came soon after opposition leader Nikol Pashinyan was released from detention. Mr Pashinyan had been arrested on Sunday after he called for Mr Sargsyan's resignation during televised talks.
As well as Mr Pashinyan, two other opposition politicians and some 200 demonstrators were held.
Protesters chanted "Nikol, Nikol" in the streets on Monday, the 11th straight day of protests. They were joined by hundreds of uniformed soldiers, despite warnings from the defence ministry that any soldiers protesting would be harshly punished.
Mr Pashinyan congratulated the people on their "victory" following the resignation. "You have won, proud citizens of the Republic of Armenia. And no-one can seize this victory from you. I congratulate you, victorious people," he wrote on Facebook.
Mr Sargsyan had faced criticism in Armenia over his close ties to the Russian president, Vladimir Putin, who has also moved between roles as president and prime minister to maintain his grip on power.
Jubilation filled the Republic Square in Yerevan as people cracked open bottles of champagne, dancing in the fountain, hugging and waving flags.
"This is what victory feels like," said Lena, wrapped in the Armenian tricolour. "This is a day to live for, this is the day to be here to witness history!" said her husband Armen.
When Mr Pashinyan arrived at Republic Square after being released, the people chanted his name and "Victory!".
Shortly after, news broke that Mr Sargsyan had resigned and it was as if a firework of joy had exploded in the square. Many here say it is a victory for the whole nation, that the people stood up for democracy and won. And it came on the eve of April 24, which Armenians worldwide mark as Remembrance Day for the victims of mass killings of Armenians at the turn of the 20th century in Ottoman Turkey.
The events in Armenia are significant, because they demonstrate that in a post-Soviet country change is possible through a peaceful, organic, grassroots movement. The fact that the Armenian authorities showed restraint and did not use excessive force against the demonstrators is also an achievement.
But it's important to remember that the new acting prime minister is an old ally of Mr Sargsyan. Only the leadership has changed.