Automating the extraction of data from HTML tables with unknown structure

David W. Embley, Cui Tao, Stephen W. Liddle

Research output: Contribution to journalArticle

51 Citations (Scopus)

Abstract

Data on the Web in HTML tables is mostly structured, but we usually do not know the structure in advance. Thus, we cannot directly query for data of interest. We propose a solution to this problem based on document-independent extraction ontologies. Our solution entails elements of table understanding, data integration, and wrapper creation. Table understanding allows us to find tables of interest within a Web page, recognize attributes and values within the table, pair attributes with values, and form records. Data-integration techniques allow us to match source records with a target schema. Ontologically specified wrappers allow us to extract data from source records into a target schema. Experimental results show that we can successfully locate data of interest in tables and map the data from source HTML tables with unknown structure to a given target database schema. We can thus "directly" query source data with unknown structure through a known target schema.

Original languageEnglish (US)
Pages (from-to)3-28
Number of pages26
JournalData and Knowledge Engineering
Volume54
Issue number1
DOIs
StatePublished - Jul 2005
Externally publishedYes

Fingerprint

HTML
Data integration
World Wide Web
Ontology
Websites
Query
Wrapper

Keywords

  • Data integration
  • Document-independent extraction ontologies
  • Table understanding
  • Wrapper creation

ASJC Scopus subject areas

  • Artificial Intelligence

Cite this

Automating the extraction of data from HTML tables with unknown structure. / Embley, David W.; Tao, Cui; Liddle, Stephen W.

In: Data and Knowledge Engineering, Vol. 54, No. 1, 07.2005, p. 3-28.

Research output: Contribution to journalArticle

Embley, David W. ; Tao, Cui ; Liddle, Stephen W. / Automating the extraction of data from HTML tables with unknown structure. In: Data and Knowledge Engineering. 2005 ; Vol. 54, No. 1. pp. 3-28.
@article{b2213e60bb204f72a0bfaa2051e09a5c,
title = "Automating the extraction of data from HTML tables with unknown structure",
abstract = "Data on the Web in HTML tables is mostly structured, but we usually do not know the structure in advance. Thus, we cannot directly query for data of interest. We propose a solution to this problem based on document-independent extraction ontologies. Our solution entails elements of table understanding, data integration, and wrapper creation. Table understanding allows us to find tables of interest within a Web page, recognize attributes and values within the table, pair attributes with values, and form records. Data-integration techniques allow us to match source records with a target schema. Ontologically specified wrappers allow us to extract data from source records into a target schema. Experimental results show that we can successfully locate data of interest in tables and map the data from source HTML tables with unknown structure to a given target database schema. We can thus {"}directly{"} query source data with unknown structure through a known target schema.",
keywords = "Data integration, Document-independent extraction ontologies, Table understanding, Wrapper creation",
author = "Embley, {David W.} and Cui Tao and Liddle, {Stephen W.}",
year = "2005",
month = "7",
doi = "10.1016/j.datak.2004.10.004",
language = "English (US)",
volume = "54",
pages = "3--28",
journal = "Data and Knowledge Engineering",
issn = "0169-023X",
publisher = "Elsevier",
number = "1",

}

TY - JOUR

T1 - Automating the extraction of data from HTML tables with unknown structure

AU - Embley, David W.

AU - Tao, Cui

AU - Liddle, Stephen W.

PY - 2005/7

Y1 - 2005/7

N2 - Data on the Web in HTML tables is mostly structured, but we usually do not know the structure in advance. Thus, we cannot directly query for data of interest. We propose a solution to this problem based on document-independent extraction ontologies. Our solution entails elements of table understanding, data integration, and wrapper creation. Table understanding allows us to find tables of interest within a Web page, recognize attributes and values within the table, pair attributes with values, and form records. Data-integration techniques allow us to match source records with a target schema. Ontologically specified wrappers allow us to extract data from source records into a target schema. Experimental results show that we can successfully locate data of interest in tables and map the data from source HTML tables with unknown structure to a given target database schema. We can thus "directly" query source data with unknown structure through a known target schema.

AB - Data on the Web in HTML tables is mostly structured, but we usually do not know the structure in advance. Thus, we cannot directly query for data of interest. We propose a solution to this problem based on document-independent extraction ontologies. Our solution entails elements of table understanding, data integration, and wrapper creation. Table understanding allows us to find tables of interest within a Web page, recognize attributes and values within the table, pair attributes with values, and form records. Data-integration techniques allow us to match source records with a target schema. Ontologically specified wrappers allow us to extract data from source records into a target schema. Experimental results show that we can successfully locate data of interest in tables and map the data from source HTML tables with unknown structure to a given target database schema. We can thus "directly" query source data with unknown structure through a known target schema.

KW - Data integration

KW - Document-independent extraction ontologies

KW - Table understanding

KW - Wrapper creation

UR - http://www.scopus.com/inward/record.url?scp=16244404907&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=16244404907&partnerID=8YFLogxK

U2 - 10.1016/j.datak.2004.10.004

DO - 10.1016/j.datak.2004.10.004

M3 - Article

VL - 54

SP - 3

EP - 28

JO - Data and Knowledge Engineering

JF - Data and Knowledge Engineering

SN - 0169-023X

IS - 1

ER -