![]() Bots are essentially a line of code or a program that is created to perform specific tasks on a large scale. > Bots: This is a general term that refers to non-human traffic, or robot traffic that is computer generated. The jargon around web scraping is confusing - bots, web scrapers, data extractors, price scrapers, site indexers and more - what’s the difference? Allow me to quickly clarify. What are bots, web scrapers and site indexers? Which are good and which are bad? As we talk this through I’ll explain what these “bots” are, how to find them and how to manage them to better protect and leverage your travel business. Overall, I see an alarming lack of awareness around the prevalence of web scraping and bots in travel, and I see confusion around what to do about it. So I am admittedly biased, but will do my best to provide an educational account of what we’ve learned to be true about web scraping in travel - and why this is an issue every travel company should at the very least be knowledgeable about. This is a good time to note that I am CEO of a bot-blocking company called Distil Networks, and we serve the travel industry as well as digital publishers and eCommerce sites to protect against web scraping and data theft - we're on a mission to make the web more secure. Google Analytics, and most other analytics tools do not automatically remove web scraper traffic, also called “bot” traffic, from your reports - so how would you know this non-human and potentially harmful traffic exists? You have to look for it. We have the data to prove it over 30% of travel industry website visitors are web scrapers. ![]() Travel suppliers, OTAs and meta search sites are all being scraped. These questions and the challenge of web scraping are relevant to every player in the travel industry. Here’s the thing about web scraping in the travel industry: everyone knows it exists but few know the details.ĭetails like how does web scraping happen and how will I know? Is web scraping just part of doing business online, or can it be stopped? And lastly, if web scraping can be stopped, should it always be stopped? NB: This is a gust article by Rami Essaid, co-founder and CEO of Distil Networks.
0 Comments
Leave a Reply. |