Intellect at 10 : DoT Damage = X over Y seconds
Intellect above 10: Dot Damage = X over Y * (intellect bonus) seconds
Basically, the amount of damage stays the same, but the DPS suffers.
I don't think that's how int works. I tested this with a Wizard and the Corrive Siphon ability (33.33 damage over 10 seconds) and no matter my int value, I tested at 19 int and 42 int (with console commands), it ticked on application, again every 3 seconds and when the debuff expired. More int = longer duration dot = more ticks. For example, with 19 int I had +45% duration which increased the duration from 10 seconds to 14.5 seconds. The DoT ticked on the initial application of the DoT, every 3 seconds after the application and when the DoT faded. Since the time between the second to last DoT tick and the removal of the DoT was less than 3 seconds the final tick only did partial damage, 7 instead of 9. (2.5/3)*9=7.5, which got rounded down to 7.
It could work differently with other spells, though.