How do researchers find out/know what older versions of a language sounded like without recordings?

Do they just base it off the written word of the time and compare it to modern versions of the language?

Basically, yes. Two things that help a lot are orthographic (or spelling) errors (they show what letter combinations sound the same and thus are mistaken to the user of the language) and poetry (which often requires precise rhythm, so it gives some clues about the stress patterns, rhyming – which again shows which endings sound the same or at least similar and which ones are distinct, and alliteration – similar to rhyming but for the beginning of words).

Poetry is even more important, as it is often passed orally for centuries before being recorded in writing much later than it is composed, so often it can give you some insight into language much older than the one used by the scribe who’s written it down.

Then there are direct descriptions of the languages by their users themselves, like The First Grammatical Treatise for Old West Norse, which describes how sounds of the language were produced and gives minimal pairs for sounds that were distinguished by native speakers. I don’t know what sources of this kind exist for Irish, but I wouldn’t be surprised if Middle Irish for example had something similar.

And then there is comparing to modern daughter and neighbouring languages. Some sound changes are more probable than others, and some sounds tend to be borrowed from other languages more often than others, so having the knowledge of modern languages one can speculate about the sounds existing in their common ancestor.

And you also look at loanwords – the form of the word in the language that borrowed it shows you which sounds of the two languages were similar at the time the borrowing happened.

Jackson Crawford, a doctor specializing in Old Norse language and Norse myth, has a very nice video explaining exactly this subject: How do we know what a dead language sounded like?, and somehow related one: Reconstructing Languages.

Thank you those videos were very interesting and informative!