Haven't creative works been known to play around with morals, conventions, and what is socially acceptable for a very long time now?
Is it any different from illustrating battlefields or reciting poems about the unsung beauty of war?
Yet I haven't heard anything about art shaping people into crazed, eccentric, and unorthodox scum... Well, except for a few well-known names, but those characters who left their mark on the history of art were probably attracted to the medium as a result of their neurodivergence, I don't believe that art itself was to blame.
On a more real note, most loli-enjoyers I've met have been grossed out by real children. This is only anecdotal evidence, but I still believe in it strongly.
Pedos become loli-enjoyers, but I don't think loli-enjoyers become pedos. If they do, I think their issue is deeper than just consuming ecchi cgdct anime; they probably have an ever-increasing porn addiction.