Treehouse

PHP Serialize() & Unserialize() Issues

By on  

I've been working on some very large forms lately and I've come to the conclusion that creating a database scheme around them wouldn't be the best option because:

  1. My customers don't need to analyze all form submissions as a whole -- form information is simply used on a per-submission basis (like a job application, for example).
  2. Making updates to these forms would be very costly since it would take quite a bit of time to add and remove DB fields as well as update the HTML form.
  3. I'd like to revert the information into an array format just like it came in easily.

For that reason, I've been using the serialize() and unserialize() functions often. Serializing an array keeps the information in an array format, so to speak, but in one long string. Anyways, I ran into the following error when testing unserialize on some information that I had serialized:

Notice: unserialize(): Error at offset 2 of 52 bytes in file.php on line 130

It turns out that if there's a ", ', :, or ; in any of the array values the serialization gets corrupted. I've found the following fix for this issue on PHP.net:

//to safely serialize
$safe_string_to_store = base64_encode(serialize($multidimensional_array));

//to unserialize...
$array_restored_from_db = unserialize(base64_decode($encoded_serialized_string));

It's a great fix to simple problem!

ydkjs-1.png

Recent Features

Incredible Demos

  • Create a Download Package Using MooTools Moousture

    Zohaib Sibt-e-Hassan recently released a great mouse gestures library for MooTools called Moousture. Moousture allows you to trigger functionality by moving your mouse in specified custom patterns. Too illustrate Moousture's value, I've created an image download builder...

  • Duplicate the jQuery Homepage Tooltips Using MooTools

    The jQuery homepage has a pretty suave tooltip-like effect as seen below: Here's how to accomplish this same effect using MooTools. The XHTML <div id="jq-intro" class="jq-clearfix"> <h2>jQuery is a new kind of JavaScript Library.</h2> <p>jQuery is a fast and concise JavaScript Library that simplifies HTML...

  • Sliding Labels Using&nbsp;MooTools

    A week back I saw a great effect created by CSSKarma: input labels being animated horizontally. The idea is everything positive: elegant, practical, unobtrusive, and requires very little jQuery code. Luckily the effect doesn't...

Discussion

  1. Great fix indeed!!! Smart thinking!

  2. Works like a charm. I haven’t found any issue with this work-around. You’re one smart duck.

  3. Great “fix”. :-) You also may want to look into a document oriented store (e.g. CouchDB) vs. serializing data into a RDBMS. This would allow you to stay flexible when storing data, but also for later querying/analyzation.

  4. Thanks for this tip! I almost deleted all my new code related to serialization when I decided to search for solutions and found this trick.

    It’s not very clear for me how it works if it encodes string after it’s serialized and decodes before it’s unserialized, but for as long as it works I’m happy!

    Thanks again!

  5. Shaddi

    @Shimon, it’s not actually encoding the string, it’s encoding the serialization. If you encode the serialized array, you’ll have to decode the array before you unserialize it.

    Great fix!

  6. @Shaddi
    My accent was made on why it does encoding after all, when serialization is done already. I understand that if you encode string after serialization you’d have to decode before unserialization :)

  7. Thank you, I was having an issue with serialize() and unserialize() and this post solved it :)

  8. Paul

    Thanks for this solution.

    Serialize and unserialize is a godsend in lieu of sessions and forms, but could be better constructed.

    Excellent advice.

  9. Great blog you’re writing! I think the easiest way to prevent these issues is not to use serialize() function. Why not use the implode() function, it can do the same as far as i know!

  10. txyoji

    When storing very long strings in MySQL, make sure you check the length.
    MySQL will silently truncate data longer than the allotted field.

    Also, using this technique instead of storing in a database means ‘schema’ changes must be done with a php script vs using a db script to migrate data.

    • ^this! Make sure the field you’re storing the encoded & serialized string in is long enough!

  11. It works for me, but you must make sure your table field has enough varchar space eg: varchar(100) as the string is too long, and you must know how to display arrays.

  12. vishal

    Thanks mate that was Gr8 Help

  13. john

    thank you so much – i am not sure why this worked and my other code didnt, but FABULOUS!

    thanks

  14. Walid

    Thanks a lot different servers work differently, the same code wouldnt work on our new server until i implemented this

    kudos to putting this up

  15. Steve

    The real question is why PHP doesn’t incorporate this into serialize() to begin with.

  16. Nick

    @Peter:

    serialise can handle objects too. This isn’t true of implode().

  17. Dave

    I understand what Shimon is saying and i agree.

    I see that if you are moving the serialized value in and out of a database it should be encoded as base64, but the issue is unserializing a serialized array that contains a ‘ (in my case) – encoding and then decoding does not solve that issue as you get the same string back!

  18. Devdutt

    Thanks Mr. David Walsh this solved my big time problem.

  19. Praveen

    @Devdutt: This solution not solved my problem, i am serializing large data and storing into database, when unserialized not working. Using php 5.0.1. it is giving offset error. “Notice: unserialize() [function.unserialize]: Error at offset 45393 of 65533 bytes “.

    Thanks in advance

  20. @Praveen.. It could be your database field has few character space, and it store only some few serialized information. This must bring some problem during unserialize(). Try to make enough space in your field, let say varchar(2000) and see..

  21. Harinder

    Ita a great fix. But there is a problem when you save a field having space in its naming.

  22. Jankes

    @Gregory Mlay:
    best way to store this would be BLOB column, ex. if you run into character set problem your serialization will remain intact, else your data can get corrupted.

  23. Thanks for the fix. This is what I am looking for. Good to serialize data when storing into cookies.

  24. tom

    You need to escape a serialized string in a manner appropriate for your DB, just as you do for any string. For example, mysql_real_escape_string() or prepared statements in the case of MySQL.

    If you base64_encode() the serialized string then you will probably obviate escaping regardless of database it since the base64 code table uses only ASCII’s alpha, numeric, + and / characters. But that doesn’t mean it’s a good solution.

    I think base64_encode() not a good replacement for using your DB’s correct escape procedures for efficiency reasons. Sometimes code fragments found on the web will work as drop-in but are a poor substitute for understanding.

  25. David, ever had an issue where the string somehow does NOT unserialize() properly after moving servers? I have a very weird issue, where the var stored from the MySQL table will NOT unserialize, as soon as we moved from one server to another. It returns false. However if I copy-paste the serialized string, put it into a variable manually, and unserialize(), it works.. weirdest thing ever.

    • I am having a very similar issue after deploying to a production server. I tried your copy/paste technique and had the same results.

      Did you ever discover what was causing your unserialize issue after moving servers?

  26. Barbara

    Thank you very much!!!

  27. olu

    Thanks a bunch for this tip.

  28. Checked solution for several hours. Great Idea. Saved life. Thank you!

  29. Thanks for great solutions :)

  30. You saved my a lot of headache!! you rock David!
    It helped me to convert an array to a $_GET

  31. Vibha

    Thanks dude..
    Yoy r0cK..

  32. Kaustubh

    Great!!! You solved my problem……..God bless you…..

  33. latorril

    Geez! Thanks SOOOO much! I’ve been working on this for a couple frustrating hours! Now I can move on with my life.

  34. kubino

    one more THX from Czech Rep!

  35. Dr Lightman

    I agree with Shimon in this, there is no reason why adding that layer of base64 encoding AFTER the serialization occurs, should resolve a problem with the unserialization, unless, and that’s not been specified in here, the serialized data was messed up by some escaping function, encoding conversion, etc, before or after being stored in a database or similar.

    I understand that in that case, preventing the data from containing characters suscettible to escaping such as quotes, would be of help, but it’s not a general case.

  36. Excellent solution. This is definitely a good way to go. I was thinking of using the preg_replace function to replace any single quotes, double quotes, semi_colons, or colons with a set of characters that is unlikely to be used in the string. For instance:

    $variable = preg_replace(‘|\’|’, “_SINGLEQUOTE_”, $variable);
    $variable = preg_replace(‘|\’|’, “_DOUBLEQUOTE_”, $variable);
    $variable = preg_replace(‘|\’|’, “_SEMICOLON_”, $variable);
    $variable = preg_replace(‘|\’|’, “_COLON_”, $variable); $variable = serialize($variable);

    The downside to this solution is that it is a lot of work for a similar result and the unserialize function must utilize the same code but in reverse. I haven’t yet tested the two but speed can definitely be a factor in which direction you would want to go. Also, with the preg_replace solution, you must use a string that will NOT be used in the variable at all or strange results will occur. Again, thanks for sharing your solution. I believe it may be a way that I will go with my code and its certainly a clever use of the base64_encode()/decode() functions.

  37. I think this is caused by modern encoding (utf-8) and the use of old MySQL library (mysql_* functions on PHP).

    Changing MySQL API to mysqli and setting the proper encoding (mysqli::set_charset) will fix this without base64.

    Base64 would use a lot of storage space.

  38. vipul solanki

    i am having same problem as ypo havd earlier.
    i tried out your code but its not working.

  39. vipul solanki

    please send me anyother solution if you find

  40. Alix

    I never leave comment, but you really save me man. Thanks

  41. Might I throw in my 2 cents:

    json_encode() / json_decode().

    • Could be useful, but json_encode/decode doesn’t keep track of types, so everything it’s converted to a stdClass object

  42. Bilal

    Thankyou!! that was really helpful :)

  43. Chris Fav

    Thank you alot, FIXED my issue.

  44. ZAGI

    hi, I had same problem, but I found that I had magic_quotes_runtime enabled which was causing the error.
    hope this would be helpfull to.

  45. MAK

    Love you man! :D
    You’ve just won my heart :D

  46. Thank you for you help!

    Although your solution wasn’t quite enough to get it to work for me it was definitely part of the solution. I was trying to store the serialized string to my sql server db in a text field. After I realized that the ” were causing problems with unserialize and used the base64_encode I then realized that the string was being truncated when retrieving it from the db. The easiest way I found to get around this was to make these 2 changes in php.ini

    mssql.textlimit = 2147483647
    mssql.textsize = 2147483647

    They were set by default to 4096 and were commented out so remember to remove the ; too.

    Hope this helps someone.

  47. mina

    After two hours of unsuccessful efforts, I found your post and been saved!
    thanks!

  48. Thanks so much. You saved me! Stackoverflow had nothing.

  49. This is a solution, yes, but base64-encoding the string makes it about 33% larger which isn’t nice for large chunks of data. I’d rather see a solution implemented strictly for the serialize() function, such as the one suggested by ‘suman’ (http://www.php.net/manual/en/function.unserialize.php#107886):


    $auctionDetails = preg_replace('!s:(\d+):"(.*?)";!se', "'s:'.strlen('$2').':\"$2\";'", $dataArr[$i]['auction_details'] );
    $auctionDetails = unserialize($auctionDetails);

  50. After 3/4 of a day of banging my head against my monitor, this post helped me solve my problem roundtripping a serialized object to a database. Thanks so much David.

  51. Kaustubh

    hi,
    I am using base64 encode and decode with serialize and unserialize. I came across following error

    Error: Notice: unserialize() [function.unserialize]: Error at offset 2 of 49151 bytes

    Some starting bytes before using base64_decode: czo1ODY4MzoiDQoJPHRhY

    After using base64 starting bytes are: s:58683:”

    • Kaustubh

      Problem solved!

      Some data truncated by mysql due to large size of data.
      Sorry!!!
      It works!

  52. Saved my ass on that one!! Thanks a bunch!

  53. This not only solves problems serializing data with quotes and stuff, but also is a solution if you want to save an array with different kinds of data (text and binary, for instance) and don’t want to worry about corruption somewhere along the way.

    Yes, the downside is your data increases in size when encoding, but serious amounts of data should not be stored in serialized format in a database anyway. For small amounts, it’s really not a big issue and a great solution.

  54. Dear All,

    i have found that the serialize value stored to database is converted to some other way format. Since the serialize data store quotes marks, semicolon, culry bracket, the mysql need to be save on its own, So it automatically putting “backslash(\)” that comes from gpc_magic_quotes (CMIIW). So if you store a serialize data and you wanted to used it, in the interface you should used html_entity_decode() to make sure you have the actual format read by PHP.

    here was my sample:
    $ser = $data->serialization; // assume it is the serialization data from database
    $arr_ser = html_entity_decode($ser);

    nb : i’ve try it and it works and be sure avoid this type to be stored in tables (to risky). this way can solve the json format stored in table too.

  55. ups my mistake. the code i’ve post before should be like this:

    $ser = $data->serialization; // assume it is the serialization data from database
    $arr_ser = unserialize(html_entity_decode($ser));
    

    i forget to write down the unserialize function.

  56. thomas

    This solution does not make sense at all as SHIMON already pointed out. B64 encoding _after_ serializing and B64 decoding _before_ unserializing just spurious.

    Even the problem is stated incorrectly as the following will work just fine:

    $anArray = array(‘foo’ => “:;\”‘”);
    print_r(unserialize(serialize($anArray)))

    You have no idea what you are talking about

    • Patti

      Thank you THOMAS for your post.

      The solution above didnt work but the (unserialize(serialize($anArray)))
      WORKED.

      Thank you.

  57. Mohsen Elgendy

    Such a great idea, Good thinking :)

  58. Many thanks for this David – fixed my unserialisation issue just fine.

  59. Paul Sandel

    Thanks for the tip, solved for me! (hehe, that one guy called you a duck)

  60. Thank, i have fixed my problem.

  61. Kevin

    You just saved my life, I thought all my data was unrecoverable. Thank you thank you thank you!

  62. Thank you man! I was trying with urlencode: base64 is the answer!

  63. Jucip Runjo

    Thank you! Helped a lot

  64. Willem

    Thank you so much!

  65. You are so clever! this solve my problem!

    Thank you

  66. Thanks, good tip. :)

  67. Thanks, you’re a life saver. I was looking for a solution to serialize – unserialize correctly for almost 24 hours and more than 3 cups of coffee. Thanks again.

  68. 1Veertje

    Thanks! This solved my problem finally!

  69. THANK YOU !! you just saved my friday night

  70. I almost lost my life time to fix this problem. So I’m very lucky to found this method.

    Thank you very much. Have a nice day!!

  71. Martín

    I came to this today and I’ve found that the problem itself was with utf8 mbstrings so I applied utf8_decode() to the serialized string and then unserialize and the problem disappeared.

  72. Din82

    First I thought I was doing something wrong. Then I realize that something wasn’t working well. This code is the true fix for complex arrays! Thank you

  73. I must have done something wrong but the base64_encode base64_decode approach is not working for me. I am still getting errors.
    dml

  74. […] are several articles that provide solutions. The most popular is to use the base64_encode() function around the serialized data. This will prevent the data from getting corrupted since base64 […]

  75. […] are several articles that provide solutions. The most popular is to use the base64_encode() function around the serialized data. This will prevent the data from getting corrupted since base64 […]

  76. Marco

    Wrong

    Ther’s no corruption!
    In some cases data export between databases or files messes up with quotes and charset.
    Internally generated and stored data will never corrupt.

    serialize/unserialize is the FASTEST way to store structured data with PHP, adding json or base64 will result in an enormous loss of performance

    • That’s actually incorrect – JSON is faster than serialize/unserialize in recent PHP versions.

    • Marco

      So wrong… the mean serialize encoding is twice faster than JSON… do some tests and see

  77. Chris K.

    To quickly explain the solution mentioned in this article, since it did not make any sense to myself at first base64_encoding data after it has been already serialized as mentioned in some comments.

    The array length issues occur when you are storing data with foreign characters in serialized arrays in your database.

    The UTF-8 value of ‘?’ is ’3f’, while the value for ‘Æ’ is ‘c3 86′. ‘?’ translates into s:1:”?”; while ‘Æ’ translates into s:2:”Æ”;. Notice the 2 replacing the 1 in the string length. So basically, what’s happening is that when php serializes the data it is storing the foreign character as a double the length (double-byte) but when it’s passed to MySQL, when the table is not formatted in UTF-8, the database converts the character to a ?, which is then stored as a single byte character. But the serialization length is not updated, so when you go and unserialize the data there is an offset error.

    Using base64_encode() function around the serialized data will prevent the data from getting corrupted since base64 converts the data to ASCII which any database collation can handle.

    You can also set your table DEFAULT CHARSET=utf8; AND/OR collation to utf8_general_ci or utf8_unicode_ci and that will solve your problem as well.

    Hope it helps others to better understand this solution.

  78. Cam T.

    I’ve been banging my head all over the place trying to figure out why I couldn’t get this to work using just serialize/unserialize. I decided to use your fix but with json_encode/json_decode as it benchmarked a tad better than serialize/unserialize

Wrap your code in <pre class="{language}"></pre> tags, link to a GitHub gist, JSFiddle fiddle, or CodePen pen to embed!

Use Code Editor