Encountering an HTTP error can disrupt your web scraping or automation tasks, and HTTP error 415 is one such issue that indicates a problem with the type of data being sent.

In this article, we’ll explore what HTTP 415 is, the common causes behind it, how to replicate it, and whether it could be used as a blocking mechanism.

Key Takeaways

Fix HTTP 415 errors by configuring correct Content-Type headers that match server expectations, preventing data format mismatches in web scraping requests.

Configure Content-Type headers to match server-expected media types (JSON, XML, form-data)
Inspect browser network requests to identify correct Content-Type for POST/PUT endpoints
Implement header validation to prevent 415 errors in automated scraping workflows
Detect anti-scraping blocks when 415 errors occur on GET requests or inconsistently
Use specialized tools like ScrapFly for automated header management and anti-blocking features
Debug 415 errors by comparing successful browser requests with scraper implementations What is HTTP Error 415?

HTTP error 415 Unsupported Media Type, occurs when the server refuses to process a request because the format or media type of the data being sent is not supported. For example, if you try to send JSON data to an endpoint that only accepts XML, you would encounter a 415 error.

What are HTTP 415 Error Causes?

The most common cause of a 415 error is sending data in an unsupported format. This can happen when the Content-Type header which tells the server what type of content is being sent does not match the type that the server expects.

For instance, if you’re sending a POST request with application/xml but the server expects application/json, you’ll trigger a 415 error.

Practical Example

Let’s explore how to configure headers, specifically Content-Type headers, in common tools like python’s httpx library, and cURL.

curl -X "POST" -H "Content-Type: application/json" https://httpbin.dev/json

import httpx

url = "https://httpbin.dev/json"
headers = {
"Content-Type": "application/json",
}

response = httpx.post(url, headers=headers)
print(response.status_code)
print(response.text)

const url = "https://httpbin.dev/json";
const headers = {
"Content-Type": "application/json",
};

fetch(url, { method: "POST", headers })
.then(response => response.json())
.then(data => console.log(data))
.catch(error => console.error('Error:', error));

use reqwest::header::{CONTENT_TYPE};
use std::error::Error;

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let client = reqwest::Client::new();
let response = client
.POST("https://httpbin.dev/json")
.header(CONTENT_TYPE, "application/json")
.send()
.await?;

println!("Status: {}", response.status());
println!("Body: {}", response.text().await?);

Ok(())
}

package main

import (
"fmt"
"io/ioutil"
"net/http"
)

func main() {
client := &http.Client{}
req, err := http.NewRequest("POST", "https://httpbin.dev/json")
if err != nil {
fmt.Println("Error:", err)
return
}
req.Header.Add("Content-Type", "application/json")

resp, err := client.Do(req)
if err != nil {
fmt.Println("Error:", err)
return
}
defer resp.Body.Close()

body, _ := ioutil.ReadAll(resp.Body)
fmt.Println("Status:", resp.Status)
fmt.Println("Body:", string(body))
}

require 'typhoeus'

url = "https://httpbin.dev/json"
response = Typhoeus.post(url, headers: {
"Content-Type" => "application/json"
})

puts "Status: #{response.code}"
puts "Body: #{response.body}"

<?php
$url = "https://httpbin.dev/json";
$client = new \GuzzleHttp\Client();
$response = $client->request('POST', $url, [
'headers' => [
'Content-Type' => 'application/json',
]
]);

echo "Status: " . $response->getStatusCode() . "\n";
echo "Body: " . $response->getBody();

In the examples above, the client is specifying that the body content is in application/json format. If the server expects a different media-type than the one that the client specified, a 415 error might occur.

To avoid 415 errors, ensure that your Content-Type header is set appropriately for the endpoint your are sending data to.

Note that many HTTP clients either do no set Conten-Type header or set it to plain/text by default which is the most common cause of the 415 status code.

415 in Web Scraping

Http status 415 in web scraping is usually encountered when scraping POST or PUT type endpoints like search queries or form submissions. For these cases it’s important to set the correct Content-Type header that not only matches the sent content type but the type server expects. To verify what content type the server expects, you can use Browser Developer Tools and inspect browser requests.

Another posibility is that the server is blocking your scraper requests and returns status code 415 purposefully to block your scraper. This is quite rare but here are some indicators that http code 415 is a block:

415 is returned on GETorHEADrequests
415 error cannot be replicated for the same identical requests

If you suspect that you are being blocked take a look at our guide on web scraping blocking or try Scrapfly Web Scraping API.

Power Up with Scrapfly

ScrapFly provides web scraping, screenshot, and extraction APIs for data collection at scale.

Anti-bot protection bypass – scrape web pages without blocking!
Rotating residential proxies – prevent IP address and geographic blocks.
JavaScript rendering – scrape dynamic web pages through cloud browsers.
Full browser automation – control browsers to scroll, input and click on objects.
Format conversion – scrape as HTML, JSON, Text, or Markdown.
Python and Typescript SDKs, as well as Scrapy and no-code tool integrations.

It takes Scrapfly several full-time engineers to maintain this system, so you don’t have to!

Summary

HTTP 415 errors occur when the data format or media type is not supported by the server. While this error usually results from incorrect content types, it’s important to consider the possibility of blocking. With Scrapfly’s advanced scraping tools and IP rotation, you can bypass such blocks and continue scraping without interruptions.