The ban will include flags, armbands, T-shirts, and will extend to posting the symbols online.
But under the new legislation, to be introduced to parliament next week, Nazi salutes will not be captured under the prohibition as they fall under the jurisdiction of the states and territories.
Selling Nazi memorabilia will be made an offence, and will be banned in retail and online stores.
The proposed law will not prevent private ownership, including items taken as war souvenirs, or artefacts displayed in museums.
But anyone seeking to pass on items will not be allowed to accept payment in return.
Attorney-General Mark Dreyfus said the new laws will send a strong message to those seeking to profit from the “evil symbols” or use them to promote hatred.
“There is no place in Australia for symbols that glorify the horrors of the Holocaust,” he said.
“The Albanese government is sending the clearest possible signal to those who seek to spread hatred, violence and anti-Semitism that we find these actions repugnant and they will not be tolerated.”
Executive Council of Australian Jewry chief executive Peter Wertheim said while the body welcomed the move, it had expressed concern to Mr Dreyfus that neo-Nazi groups will get around the legislation by using other symbols including the death’s head and sun wheel.
“We have urged the government to review the legislation periodically so that if our concerns are borne out, the prohibitions may in due course be extended to the display of other Nazi symbols,” he said.
The legislation only outlaws the two Nazi Hakenkreuz and Schutzstaffel symbols.
Mr Wertheim said the council was also worried about the lack of any prohibitions on the performance of Nazi gestures such as the salute.
“This will leave a gap that will need to be filled by state and territory legislation,” he said.
Displaying symbols for religious, educational, or artistic purposes are among a range of exclusions from the ban.
It will not affect the use of the swastika for people observing Hinduism, Buddhism and Jainism.
Mr Dreyfus said the government had been responsible in taking the time to get the bill right, describing it as “well-targeted and effective”.