Posts about Screen Scraping

This is a Screen Shot of wGet

Screen Scraping Service – In The Early Days

Our first web screen scraping service project was back in Oct 2003. The market for web automation and data extraction did not really exist at this time, this market was new and only just emerging !

Screen Scraping Service

In 2003 Screen Scraping and Web Data Extraction “as a service” was unheard of ! These days however, doing a quick search on google  you will find  numerous web sites offering screen scraping services and scraping tools.

Try searching google  for any of the following :-

  • content scraper
  • screen scraping service
  • scraper software
  • website scraper software
  • data scraper software
  • scrape data from website
  • scraping data from websites
  • how to scrape websites
  • etc

In 2003 (after a search on google) we found 2 interesting tools,  wGet and iOpus Internet Macros.  These tools were some of the few utilities around that had the ability to Crawl a Web Site and then effectively Screen Scrape Data.


wGet featured in the Hollywood movie Social Network. Harvard student Mark Zuckerberg creates the social networking site that would become known as Facebook. In the movie Mark Zuckerberg  is seen using wGet to scrape data to populate content on his web site then known as “

wGet is still in use today. I noticed recently that “WSUS Offline” use wGet to download Microsoft updates. If you ever experience problems updating a Microsoft operating system via the built-in “Windows Update” tool, you need to visit  ! This tool can be used online or offline  to update windows machines.
It’s good to see wGet is still being used 9 years on.

This is a Screen Shot of wGet, use wGet for Screen Scraping Service

This is a Screen Shot of wGet

iOpus Internet Macros

iOpus Internet Macros has the ability to record web browser input. Once the scripts are written they can be replayed over and over doing hours a repetitious work. iOpus Internet Macros programmatically interacts with websites. It fills out forms and automates the download and upload of text, images, files and web pages. It can import or export data to and from web applications using CSV & XML files, databases, or any other source.

iOpus Internet Macros Photo Of CD, use iOpus Internet Macros for Screen Scraping

iOpus Internet Macros Photo Of CD


Competitor Price Watch

Competitor Price Watch

Competitor Price Watch


In order for you to be a successful business person, you need to ensure that you are always up-to-date on the latest information regarding your company. More importantly, though, you always have to keep a close eye on your competition. By using the latest in web mining software, we can help you accomplish this.


Who Are We?


We are iHarvest, and competitor price monitoring is our specialty. We are a team of elite programmers and software engineers with years of experience using various web data extraction techniques to give our clients helpful information that they can use to build their businesses around. We use here to the highest standard of customer service, and we will be happy to answer any questions you may have regarding how our service works and how you can use it to maximise your revenue.


How does web data extraction work for Competitor Price Watch ? 


We visit the entire span of your competitions’ web content, taking note of all information that can be useful to you and your business. Basically, you supply us with an idea of what domains we should monitor as well as what public information on those domains you would like to have up-to-the-minute access to. For example, if you own a car dealership, it would be beneficial for you to know the sticker prices of cars at other local dealerships so that you know how to stay competitive. In this case, we would monitor the most recent sticker prices advertised on car dealerships in your area and send you summarised accounts of the information at your request.


Prices are just one example of the type of information requested by our clients. We also track various other quantities such as number in stock, number sold and many other variables. It is important to note that we are not a “hacking” service, i.e. we only can provide you with large quantities of strictly public information. With that said, we are able to cover an enormous amount of data in frequently updating intervals that would not be possible without the use of specialised software.


How Can I Benefit from Using Web Mining Software for Competitor Price Watch ? 


That is entirely up to you. By knowing how the larger pool of your industry as a whole is functioning, you have a larger sample from which you can make statistical inferences. Going back to the car dealership example, you may notice that the dealership downtown has been selling lots of red cars lately. From this, you conclude that red cars are in high demand, and you adjust your prices, shipping orders and marketing strategy accordingly to ensure that you take advantage of the latest trends in the market. By increasing your scope of the market as a whole, you are better able to make decisions going forward.


What about Your Competitors? How Do I Know That You Can Offer Me the Best Service?


As was mentioned earlier, we have a reputation for maintaining the highest level of service to our clients. If for any reason you are unsatisfied with our information service, let us know and we will do everything we can to make sure that we make whatever adjustments are necessary for you to have a positive experience.


Web mining software is being used by thousands of online services and retailers worldwide, and the number continues to grow every single day. It could be entirely possible that someone in your industry is already monitoring your web activity with web data extraction. Make sure that you don’t get left behind by allowing your business to resort to dated research methods. Web extraction is how businesses are and will continue to assess each other to ensure that they are able to stay competitive in an increasingly ferocious market.


Try using our web mining software today for your business and see what you have been missing from the big picture all along. We are absolutely certain that when you see just what kind of advantage and power exists in web mining for business owners, you will continue to come back to us for all of your competitive research needs!

Competitor Price Watch

Competitor Price Watch


See also : Competitor Price Analysis for more information about Competitor Price Watch