It's worthwhile to read some of the extra-biblical stuff out there as a way to learn about history, language, and the thinking of people at the time.
For example - Josephus has a ton of context and insight that's valuable... although non-scriptural... For me, it provided some insight on a lot of things Jesus said and warned about... like, say WHY Jesus preached nonviolence against Civil authorities....
The danger in it is that it's NOT canonical... Not inspired for a variety of reasons.... Like, for example - a book may have started inspired, then ended up being "Edited" by heretics.. Or, it may have just been uninspired to begin with - but managed to hang in there for a *LONG* time... OR... There may be no way to ensure the continuity of the original text - through 2,000+ years of transcription.
As such - you REALLY have to be on your guard, because so many of them are so subtly crafted...
For example - I read an early Christian writing called the "Didache".. It *Seems* like good and spiritual information at first, and then you think about it.... Subtle commands that would ensure you would mistreat or reject a true Apostle or Prophet.. Subtle direction that ends up having you turn out Godly men.. An overemphasis on legalism and moralism, etc...
The trouble we run into is that we do not learn how to detect error by studying nonscriptural and heretical works... but rather we learn to detect error by becoming intimately familiar with The Genuine Article.
Thanks