Sly was exceedingly tough for my first play through at level 5-6, i barely survived after half a dozen retries. However, on the second playthrough I did it at the same level with much easier results --- the predominant factor was using figurines for the 2nd time.
Just having the wooden beetles figurines (can be found at the theater in copperlane, hidden), and phantom figurine (in a locked chest in vailian embassy, potentially need a few lockpicks to open if your level is low).
Starting battle to summon one set of creatures near the wizard, and another set of creatures near the priest (i have the figurines on two frontliners), then just task the beetles and phantoms to attack the casters. Since there are 6 creatures summoned, at least half or more will focus on those creatures and the creatures will kill the caster very quickly. This massively reduce the amount of damage your party take, you can methodically using your DPSer to take out sly and other heavy hitters (rogues, rangers etc).
It was a landslide victory my off-tank (a dps focused monk) barely lost 1/4 of health and rest of my party was almost completely unscathed.