Data Harvesting

Harvest new and modified data from public websites

Website Harvesting and AI Transformations That Deliver Structured Data to Your Systems 

Organizations need to harvest and structure data and content posted and maintained on public websites. Websites are often the version of record for policy, procedure, legal, and regulatory content. Many businesses benefit from daily robotic scans of updated website content with structured XML feeds back into internal systems.

The volume and complexity of this type of information means that manual approaches are slow, error prone, and cost prohibitive. We provide automated website scraping configured to your business needs with customized XML feeds back to your organization. 

Tools: Data Harvester, GATE, Lucene tokenizer, JAVA, JAPE, PERL,  TensorFlow

A deeper solution beyond simple website scraping

DCL provides a truly useful solution that goes beyond web scraping. DCL has developed methods and bots to facilitate high-volume data retrieval from hundreds of websites, in a variety of source formats (HTML, RTF, DOCX, TXT, XML, etc.), in both European and Asian languages. We produce a unified data stream that is converted to XML for ingestion into derivative databases, data analytics platforms, and other downstream systems. This process of normalization and transformation of content to automate import into a customer’s business system maximizes business value. A key to successful projects is the depth and quality of up-front analysis to ensure complete and accurate results.

DCL Data Harvester comprises

  • Filtering programs

  • Downloading handler

  • Metadata gatherer

  • File differencing programs

  • Natural Language Processing programs

  • Data and content transformation programs

  • Secure repository

DCL’s solution harnesses technology in Natural Language Processing and Machine Learning to help enable solutions powered by Artificial Intelligence. With sophisticated automated processes, DCL optimizes content to collect information, streamline compliance, facilitate migration to new systems and databases, maximize reuse potential, and ready it for delivery to all outputs.

Mark Gross, President, DCL

WEB SCRAPING: SCIENCE OR ART?

RELATED CASE STUDY

A major financial institution selected Data Conversion Laboratory to accurately track financial compliance requirements across hundreds of jurisdictions. 

[READ MORE]

WHAT OUR CUSTOMERS SAY

I've got personal experience with DCL. Their Business Development department is very responsive, and their reputation is stellar. I've never heard anyone in my business  have a bad word to say about them. Highly recommended

DCL Data Harvester is an ideal website scraping solution for all industries that rely on regulatory and compliance management data as well as keeping up to date with constantly changing website content. DCL conducts upfront human analysis of target websites and content to ensure your content and metadata are captured, structured, and complete.

Industries Served

Shield_3x.png
Library_3x.png
Graduation_3x.png
Book_3x.png
Group_3x.png
Medicine_3x.png
Scales_3x.png
Settings_3x.png
  • DCL LinkedIn
  • DCL Twitter
  • DCL YouTube

61-18 190th Street, Suite 205

Fresh Meadows, NY 11365

+1 718.357.8700

info@dclab.com

HOME  /  INDUSTRIES  /   SOLUTIONS  /  SERVICES  /  RESOURCES /  ABOUT  /  CONTACT  /  PRIVACY  /  TERMS OF USE

© 2020 Copyright Data Conversion Laboratory, All Rights Reserved.