It's good that Affinity has some rudimentary image tracing capability now, but it's very far from being great. The options available for adjusting the trace are few indeed, and whether you get a decent result or not seems to be mostly down to luck.
Fortunately, there are a couple of free alternatives that do a much, much better job.
Inkscape is one, though its tracing dialog isn't really very intuitive. Still, once you figure out how to use its many options, it can create some excellent tracing results.
VectorCraft is another. It's also an option-heavy tracing system, but it seems to me to be a bit easier to get around, and it does a very good job indeed. I like that it creates a stack of overlapping vector objects, rather than every object punching a hole in the one beneath. However, if you prefer the latter, that is an option that can be chosen.
Both of these will produce SVG files that can be opened in Affinity. VectorCraft has its own proprietary file format as well; I'm not familiar enough with it yet to know what its other file export capabilities are.
CorelDraw is pretty far from free, but it's the vector illustration app I grew up with, and even the ancient CorelDraw X7 I have installed will do a much better, more flexible job of tracing than Affinity.
So, there are options. It would be nice, however, if Affinity had a worthwhile tracer that could be used within the program, rather than having to work around its limitations by dropping out to another system.