Savers are losing an estimated £4.3 billion a year in total from having money languishing in poor-paying accounts, research from Which? has found.
The consumer group has launched a new campaign pressing banks and building societies to "scrap the savings trap", by displaying interest rates more clearly on statements and move people out of "zombie" accounts, which are closed to new customers and often pay rotten rates of interest.
Which? said its own consumer research found that three-quarters (75%) of people do not think banks do enough to help savers get a good deal and one third (35%) have not switched their main savings account because they do not think it would make a difference.
It said analysis of Bank of England savings data and its own research suggests there is a difference of £4.3 billion a year between the average interest paid to UK savers in savings accounts and cash Isas compared with the interest they would earn if their money was being saved in a top-paying account.
It said that in March, of the 1,999 easy access savings accounts and cash Isas on the market, 82% were zombie accounts that were no longer available to new customers.
More than one third (39%) of accounts closed to new customers paid 0.5% interest or less and 16% paid 0.1% or less.
Which? argues that the savings market can be "confusing", with some accounts paying very different rates of interest having quite similar names.
It said that, for example, for a balance of £5,000, Halifax's "reward saver" account, pays up to 1.70%, but its "saver reward" pays up to 0.1%.
The consumer group wants to see banks closing zombie accounts and moving people's money into one default easy-access or Isa account at the end of fixed terms.
It said rates of interest should be displayed prominently and consistently on all statements, annual summaries and online pages. People should also be told more clearly when a bonus rate or fixed term on an account is coming to an end, it said.
Which? executive director Richard Lloyd said: "With many savers never switching because they don't think it will make a difference, savings providers should do more to help their customers get the best deal.
"They need to be clear about interest rates, let people know when bonus rates come to an end and make it easier for people to switch Isas.
"Banks and building societies must scrap the savings trap and free savers from poor-value accounts."
Economic Secretary to the Treasury, Andrea Leadsom, said: "The Chancellor announced a number of measures to help and support savers, notably increasing the cash Isa limit to £15,000, at this year's Budget. These changes will give savers more flexibility and choice.
"The study published by Which? today is an important piece of work and I hope it galvanises the industry to make sure they are playing their part to support savers.
"The Government is determined that the users of financial services are able to get the best possible deal, whether it be through switching current accounts or being able to access the best possible savings product for them."