Teachers discern too wide a gap between the two exams for potential candidates to move up to National 5, according to a letter by the Mathematics Advisory Group Scotland to Mr Swinney.
As a result, “hard-working candidates” who want to develop their maths skills are often excluded, the group said.
In more than a quarter of National 5 maths entries last summer, pupils gained no award, against only eight per cent for those taking English, SQA figures show.
The advisory group also cited the scrapping of internal classroom assessments – one of the measures announced by Mr Swinney in September to help cut teacher workload – as a factor. The SQA subsequently decided to integrate some elements into tougher final exams or other coursework.
However, confusion has followed over the content of the exams and the removal of a part of the qualification whereby candidates could gain a National 4 if they failed at National 5.
Taken together, the changes are causing perceptions among pupils that maths should be avoided as a “difficult” subject, the advisory group said.
Last week the Scottish Secondary Teachers’ Association urged Mr Swinney to hold an urgent review of the National 5 changes. Almost two-thirds of members surveyed believed the SQA changes would result in a bigger workload.
An SQA spokesman told the Times Educational Supplement Scotland: “We welcome and regularly seek the views of teachers. National 5 is within the parameters of difficulty which we would expect for subjects at this level.”
The spokesman said perceptions of maths being more difficult than other subjects could also be attributable to teacher methodology, level of attainment during the first three years of secondary and how much teaching time was allocated to maths.
A Scottish government spokesman said: “We always listen to the views of teachers and changes to the qualifications represent a considerable reduction in workload.
“Thousands of professionals were involved in the development and refinement of the qualifications, including maths teachers.
“The Scottish government, SQA and Education Scotland are continuing to work with them, to ensure that our young people have the best possible opportunities to attain to the highest level possible.”