DOMDocument and UTF-8 Problem
A few weeks back I shared how I used PHP DOMDocument to reliably update all image URLs from standard HTTP to HTTPS. DOMDocument made a difficult problem seem incredibly easy ... but with one side-effect that it took me a while to spot: UTF-8 characters were being mutated into another set of characters. I was seeing a bunch of odd characters like "ãç³" and"»ã®é" all over each blog post.
I knew the problem was happening during the DOMDocument parsing and that I need to find a fix quickly. The solution was just a tiny bit of code:
// Create a DOMDocument instance
$doc = new DOMDocument();
// The fix: mb_convert_encoding conversion
$doc->loadHTML(mb_convert_encoding($content, 'HTML-ENTITIES', 'UTF-8'));
After setting the character set with mb_convert_encoding, the odd characters vanished and the desired characters were back in place. Phew!
![Responsive Images: The Ultimate Guide]()
Chances are that any Web designers using our Ghostlab browser testing app, which allows seamless testing across all devices simultaneously, will have worked with responsive design in some shape or form. And as today's websites and devices become ever more varied, a plethora of responsive images...
![Detect DOM Node Insertions with JavaScript and CSS Animations]()
I work with an awesome cast of developers at Mozilla, and one of them in Daniel Buchner. Daniel's shared with me an awesome strategy for detecting when nodes have been injected into a parent node without using the deprecated DOM Events API.
![CSS Vertical Centering]()
Front-end developing is beautiful, and it's getting prettier by the day. Nowadays we got so many concepts, methodologies, good practices and whatnot to make our work stand out from the rest. Javascript (along with its countless third party libraries) and CSS have grown so big, helping...
![TextboxList for MooTools and jQuery by Guillermo Rauch]()
I'll be honest with you: I still haven't figured out if I like my MooTools teammate Guillermo Rauch. He's got a lot stacked up against him. He's from Argentina so I get IM'ed about 10 times a day about how great Lionel...
instead of converting the input string from your encoding into UTF8 you can also tell the DOMDocument with the 2nd arg in which encoding your string is.
http://php.net/manual/en/domdocument.construct.php
This should save you some cpu-cycles and reduce memory consumption.
Awesome, thank you for pointing this out!
Actually I was already doing
and still getting weird characters like  – adding your fix did the trick for me, thanks David!
It turns out that specifying the encoding argument when you instantiate a
DOMDocumentdoesn’t encode the contents, just sets the document’s header – see this comment http://php.net/manual/en/domdocument.construct.php#78027. It probably wasn’t necessary to specifyUTF-8anyway, as that’s the default. So something like David’s fix is needed to change any unwanted encodings in the content.Thanks for the tutorial. It really saved time.
Thank you very much you life saver
Thank you very for this :D
Caught this bug too, even after instantiating it properly
Awesome, character encoding has always been a pain the a.
Thank you very much David.
This helped me a lot. because I had bad characters in wordpress the_content.
This method removed special characters pasted from word document.
:D
Thanks a lot, David.
I was about to pull my hair out :D
Whilst this sorts out the worst of the character nasties, I am still seeing instances where an apostrophe appears as a question mark.
The character on the input end is ’ … it appears as ?
Somtimes I think what I may be experiencing is the crime of taking articles which originated in Microsoft Word, or maybe even Google Docs – ie. curly apostrophe. But I’m not sure – there are pieces of text that I’ve written out directly, which I’m not sure could explain this problem.
It’s frustrating that I can’t iron this out. They appear just fine if I disable my code which uses DOMDocument.
No code that I can find online (not MS Word character-conversion functions https://stackoverflow.com/questions/1262038/how-to-replace-microsoft-encoded-quotes-in-php and not DOMDocument modifiers) seems to work.
In PHP8.2, mb_convert_encoding is deprecated. Instead of
I am now doing this:
Has been working fine in my tests