After restarting a game to get level scaling on, I noticed it didn't seem to make much of a difference, so I decided to try to figure it out using the console.
I made 3 new characters, all the same stats(18 might/dex/per fighters, tanked resolve, few passives taken other than defensive stuff, 1H damage using axe, most of the other stuff is random crap that does nothing).
On one character, I used Berath blessing and was lvl 4. This was the non scaling character. As expected, most enemies were tough to beat solo(I left companions out of the tests) and getting to the panthers encounter in the digsite was an instant wipe in seconds.
On the 2 other characters, I used the console to level to 15. On one I selected scale all, on the other I selected scale all only up. I did both because when I did only up I couldn't tell if it did anything and thought maybe only up was bugged. The result is basically what you'd expect from a lvl 15 walking in a low level area, I was getting hit a lot less, with many more grazes, although I was still getting hit. I was killing stuff a decent bit faster. I could solo the panther encounter, but losing almost all my life(that's without using any skills, without optimal choices of abilities/passives and with the start mail armor+1h axe no shield setup and cleaving stance mind you, I wouldn't lose any health activating buffs and actually playing the game).
Monsters have the same defenses(including deflection) on the tooltip, they have the same health(60-61 health on the boar on the beach for example). The only thing that they might have more of is accuracy so they can't still hit you, but I'm not even sure of that since again, was getting hit a lot less. I made sure to check panthers during daytime on both(at night time they get +10deflection).
Is level scaling still done "by act" even though the act aren't visible. So for example I won't trigger a level scaling check until I reach act2, wherever that is(neketaka queen maybe?) so everything that's in "act 1" doesn't scale up? But how does the scaling down work then, why does that part work(I assume, but I haven't seen anyone complain about it) but not the other way? It seems to me that level scaling upwards just doesn't do anything. That's a large part of why the game is boringly easy.