As for Old Norse vs modern Icelandic and other Scandinavian languages, Dr. Jackson Crawford made a
great video about it: Old Norse and the Modern Scandinavian Languages (and a few more on related subjects:
What's the "realistic" Viking language?,
Old Icelandic vs. Old Norwegian, Old West Norse vs. Old East Norse,
Earlier and Later Old Norse) – he compares features of modern languages with three dialects/periods of Old Norse (older eastern and western dialects, and later Old Icelandic, which is the best known, ’cause it’s the language of the written sagas).
As for the reason – well, all languages evolve, so it’s really not surprising that Icelandic did deviate phonetically. It is a bit surprising that it did not evolve as much grammatically, and that western Norwegian dialects did not evolve that much phonetically (although, they did have some sound changes too).
As CaoimhínSF wrote, Icelandic had a fairly big influence from Goidelic peoples, modern Icelandic for example has a progressive construction very similar to Irish (and somehow to English), but absent from Old Norse or any other Scandinavian languages (well, maybe present in Faroese?), eg. ‘I am burning your house’ would be
ég er að brenna husið ðitt in Icelandic, lit. ‘I am to/towards burn your house’, syntactically somehow similar to
táim ag loscadh do thí (I am
at burn
ing of your house) – this construction might be a grammatical borrowing from some Goidelic dialect. In Old Norse, depending on the dialect, this would be
ek brenni hús þitt (west) or
jak brenni þitt hús (east), modern Norwegian nynorsk
eg brenner huset ditt, Swedish
jag bränner ditt hus.
Phonetically modern Icelandic for example diphthongized long vowels (eg. á is /au/, æ is /aɪ/, é is /je/), merged old front /ø/ (i-umlaut of o) and back /ǫ/ (u-umlaut of a) into a single front vowel written ö, and changed
ll (long /lː/?) to /tl/, while some western Norwegian dialects, as I believe (but am not certain, cannot quickly find a sensible source on that), retain single vowels and all retain /l/ and a distinction between front ø and back o. On the other hand, many Norwegian dialects do have palatalization of some consonants before front vowels, often more prevalent than Icelandic, similar to Swedish (eg. they palatalize older /skj/ and /sj/ to /ʃ/, /kj/ to /ç/), and also ones that do diphthongize old long vowels… So perhaps claiming that western Norwegian dialects are most conservative phonetically isn’t that accurate either. It also seems, Danish, on the other hand, has no palatalization (so retains /ski/, /s/, /k/, /g/, etc. before front vowels).
EDIT: so to make a tldr – Icelandic did innovate a lot in vowels and diphthongs, while not that much in consonants, Norwegian is much more conservative in vowels, while having palatalizations which changed consonants quite a lot.