Set the User Agent With PHP cURL

By  on  

A few months back, I shared with you how to download the contents of a URL and execute a HTTP POST transmission using PHP cURL. For security purposes, some hosts require that a common user agent be present in the POST. If an unacceptable user agent is given, the POST is ignored. Luckily, cURL allows us to "spoof" the server using any user agent we choose:

curl_setopt($ch,CURLOPT_USERAGENT,'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13');

A little sneaky but it could get you out of a jam at some point!

Recent Features

  • By
    5 HTML5 APIs You Didn’t Know Existed

    When you say or read "HTML5", you half expect exotic dancers and unicorns to walk into the room to the tune of "I'm Sexy and I Know It."  Can you blame us though?  We watched the fundamental APIs stagnate for so long that a basic feature...

  • By
    Regular Expressions for the Rest of Us

    Sooner or later you'll run across a regular expression. With their cryptic syntax, confusing documentation and massive learning curve, most developers settle for copying and pasting them from StackOverflow and hoping they work. But what if you could decode regular expressions and harness their power? In...

Incredible Demos

  • By
    CSS 3D Folding Animation

    Google Plus provides loads of inspiration for front-end developers, especially when it comes to the CSS and JavaScript wonders they create. Last year I duplicated their incredible PhotoStack effect with both MooTools and pure CSS; this time I'm going to duplicate...

  • By
    Duplicate the jQuery Homepage Tooltips Using MooTools

    The jQuery homepage has a pretty suave tooltip-like effect as seen below: Here's how to accomplish this same effect using MooTools. The XHTML The above XHTML was taken directly from the jQuery homepage -- no changes. The CSS The above CSS has been slightly modified to match the CSS rules already...

Discussion

  1. Thanks for this tip. Was working on a project and found your tip very handy.

  2. Thanks for this! Was needed for a wikipedia curl request.

  3. Thanks for this, it for me too.

  4. hi..
    How to check whether a user agent string is supported by all the urls or not ? I have tried many urls with different user agent strings but I couldn’t find any user agent which is supported by all urls. When a user agent string is not supported, I get 0 (zero) as http response code and null as mime type. Please help.

  5. Thanks, without setting user agent I was getting this error:

    at nl.bitwalker.useragentutils.Browser.isInUserAgentString(Browser.java:154)

  6. Brilliand

    “for security purposes”… that’s incredibly weak security. I think it’s more common for the website to be doing useragent sniffing (to tailor the response to the browser), and simply not bothering to support uncommon useragents.

  7. perfect – thank for the tip!

  8. MJ

    Thank, that helped!

  9. i get error code like this. curl_setopt() expects parameter 1 to be resource, null given in /home/xxxx/xxxxx

  10. Try this…

    function scrap($url)	{
    	$curl = curl_init($url);
    
    	curl_setopt($curl, CURLOPT_USERAGENT, "My User Agent Name");
    	curl_setopt($curl, CURLOPT_FAILONERROR, true);
    	curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
    	curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
    
    	$html = curl_exec($curl);
    	curl_close($curl);
    
    	return $html;
    }
    

    You are getting the error as curl_setopt() required first parameter as a resource returned by the curl_init($url);

  11. This, in fact, just got me out of a jam.

  12. Jim

    Almost a full 10 years later and this post just really helped me out! Thanks!

Wrap your code in <pre class="{language}"></pre> tags, link to a GitHub gist, JSFiddle fiddle, or CodePen pen to embed!