We switched the opposite direction (D3 to Leaflet) on a different app and got a performance speed up for big sets of vector shapes. Seems like there are tons of ways to engineer these web graphics apps and no clear "best solution".