API Endpoint Protection

How do you protect API services from automated traffic, if all the traffic hitting your API endpoint is automated? Here's where the power of VerifiedVisitors AI/ML platform for visitor management can help!

Many API providers face a large problem with Invalid Traffic (IVT) continually data harvesting their API services. High levels of IVT on APIs leads to a host of problems. Automated bots are systematically farming their API content, stripping all the content away and deploying the content in other applications, without crediting or benefiting you, the content supplier. These illegitimate “mashups” often enrich the farmed data with other data sources that are also farmed, resulting in a smorgasboard offering that is richer and more diverse than the legitimate content owners service, with none of the associated costs.

Often these bots produce a strain on the load of the API service due to the high volume of requests. Instead of just querying a set of results, these bots systematically query the entire data set, and may run at periodic intervals throughout the day.

A good example of this type of automated bot are price scrapers tools for e-commerce platforms. Consumers often look for the best prices - particularly for white goods - where there is no clear differentiation between suppliers or service levels. Competing ecommerce sites often reduce the per unit price of the item, but increase the shipping charges against their competitors, so they can claim the cheapest headline price, but increase the margin by padding the shipping costs. Automated bots systematically lookup each price per unique postal district - resulting in a vast query across the entire shipping cost database.

Existing Tools for end-point protection don't work.

To prevent automated traffic on web sites, one common technique is to use a JavaScript tag that asks a user’s browser to perform a small calculation and return a proof of work (PoW). Essentially the use of the PoW filters out the traffic coming from non-full stack browsers. Another method is to use a digital fingerprint, again in the form of JavaScript which interrogates the client and finds out a wide range of hardware and other details about the client processing unit.

These methods don’t work for API endpoint protection as the traffic on these API’s is all automated, and it makes no sense to look for a browser and associated CPU, mobile client etc. Everything hitting the API endpoint is automated traffic and the proof of work / fingerprint won’t be captured for legitimate users.

Given the lack of any tools, one general method is just to rate-limit access to the entire API. Although this helps smooth out operational peaks which can cause a server outage or maintenance problems, as well as increased bandwidth / hosting costs, this method has obvious disadvantages. Service levels are restricted for the legitimate subscribers, who often make up the vast majority of the usage base. The illegitimate bots just suck up all the remaining resource and bandwidth.

VerifiedVisitors Unique Behavioural AI Platform.

VerifiedVisitors has a unique approach to API protection. We examine the incoming API requests, and use our ML platform to detect API abuse from the behaviour of the visits alone. Although there is very little data to go on, it turns out that it's certainly enough to detect API attacks. Legitimate user don't systematically extract the database and seek to disguise their origins. The bad guys do!