So I had this idea for how I'd like the monster compendium/beastiary to work in the game:
Monster Lore There are 3 components of monster lore to each entry in the monster compendium; you are rewarded with XP for each component you unlock and gold from a quest node for completed entries. 1. Initial contact with the monster Whenever you meet a new monster you get an entry in the Monster Compendium, concept art sketches of it and a basic description. 2. Extended familiarity with the monster Discovering a monsters attributes or killing a number of them grants you another entry on the monster, detailing its strengths and weaknesses. 3. Obtaining expert research on the monster The final entry deals with the monsters behaviour and ecology or possibly some relevant history. This can usually only be obtained by finding expert field researchers or rare books on the subject.
Rare Monsters There should be a few rare monsters that can only be found through exploration to give the monster hunters something to chew on. These don't have to be completely new models, using an existing model with a different color and a different set of attributes would work fine. These monsters could only come out at night, or have some other condition that might make them harder to find.
Completion Reward Finally, you are rewarded with a special bonus from a quest node for fully completing all monster entries. It could be simple XP/gold, an item, a skill, a title, a pet or something else. I imagine it to be like a book and you flip through the pages, I think having the concept art would look really cool in this context and fit in well with the idea that your party is sketching and writing the 'book' as you go along.