Precision Recruiting
IT Jobs
Web Mining
Site Map
 [ Home
 [ Finance ]  
 [ Web Audit ] 
 [ Consulting

Web Robot Technology

A web robot (also called web crawler, web spider or knowbot) automatically browses web pages, follows links and harvests relevant information found online.
  • A simple application: downloading historical stock prices.

Web robot: sample output (Yahoo Finance)

Date	Open	High	Low	Close	Volume	
28-Jun-01	52.80	53.26	51.51	52.08	14363200	
27-Jun-01	53	53.15	52.15	52.68	10456900	
26-Jun-01	52.10	53	51.76	52.85	11302400	
25-Jun-01	53.70	53.84	52.70	53	11224600	
22-Jun-01	54.05	54.30	52.75	53.10	13431000	
21-Jun-01	52.60	55	52.30	55	26473200	
20-Jun-01	49.74	52.80	49.74	52.80	16370300	
19-Jun-01	50.40	50.90	49.53	49.84	10924200	
18-Jun-01	50.75	50.80	49.55	49.80	13073400	
15-Jun-01	49.40	50.90	48.18	50.90	27328600	
14-Jun-01	51.50	51.53	50.06	50.41	12707400	
13-Jun-01	51.95	52.89	51.50	51.55	84009000	
12-Jun-01	51.90	52.65	50.60	52.10	10597000	

What can you do with our web robot, besides downloading stock quotes?
  • Copyright infringement tracking: search the Web continuously - millions of web pages - for breaches of copyright, counterfeit goods, misuse of trademarks. See a small sample of web pages extracted and scored with the web robot.

  • Check broken links and site performance, track reciprocal links that have been discontinued or are no longer working, automatically check your ranking on search engines, monitor affiliate programs, automatically measure load time and size (in bytes) for all your web pages, find web sites that link to you.

  • Automatic Web Search, Database Queries and Data Harvesting: The web robot can be programmed to automatically query a search engine, then extract relevant information and create a simple text file on distilled data. It can also download information from Internet databases: Financial news, historical stock quotes, trading signals, weather data, lottery results, e-mail directories, etc. The robot will also download binary files (images).

  • Miscellaneous: Automate Category Assignment across entire Web-Site: The web robot can automatically assign a category to every web page on your site by mechanized keyword density analysis. It can also upload information on external Web Sites through Html forms. For instance, you can perform automatic posting in some message boards.

What our clients are doing with our web robot

  • A major internet company uses the robot to check the accuracy of product and price information published online.

  • A major search engine company has created a large database of keywords and short phrases while browsing the web with our robot. The database was used to improve search results relevancy and identify correlated queries.

  • A web hosting company in The Netherlands is monitoring trademark violations with our customized robot.

  • uses our robot to download their auctions from Ebay and upload them on their website. They also download product information (price, title, picture) on Amazon with another customized version of the robot.

  • The National Radar Facility of University of Colorado downloads webcam images automatically generated on their server, every day.

  • We are using our web robot to create a domain name database and assign rankings to each domain.

Data Mining Machine Learning Analytics Quant Statistics Econometrics Biostatistics Web Analytics Business Intelligence Risk Management Operations Research AI Predictive Modeling Actuarial Sciences Statistical Programming Customer Insight Data Modeling Competitive Intelligence Market Research Information Retrieval Computer Science Retail Analytics Healthcare Analytics ROI Optimization Design Of Experiments Scoring Models Six Sigma SAS Splus SAP ETL SPSS CRM Cloud Computing Electrical Engineering Fraud Detection Marketing Databases Data Analysis Decision Science Text Mining