I've canvassed over 30 canoes (it's just a hobby - honest!), but have only used a primer on the last 4 canoes - for all the others I just started painting on the filled canvas.
Which ever direction you go you'll want to sand down the filled (and cured) canvas. This is like sanding down sandpaper as a filled canvas surface is a little like cement. I believe what makes the surface so rough is little cotton fibers coated with 'cement' sticking up from the surface. I've found that if you apply the paint directly without a primer then it'll take at least two coats of paint, sanding between coats, to get a surface that feels & looks smooth. And the paints I use are noted for providing a hard surface, so again it's like sanding sandpaper.
What's nice about a primer is that it sands easily (it's a little soft) and does on a little thicker than paint. This makes sanding down to a nice smooth surface much easier, and being thicker it can hide slight canvassing imperfections plus cover those fibers that stick out. Also on the plus side is that primer is generally cheaper than paint.
On the minus side is that you've got a less hard surface between the canvas and the paint, so I wonder if this makes the painted surface more susceptible to deep scratching that gets down to the (whitish colored) primer.