Klareco: An Indexing-Based Architecture For Interactive Visualization Of Heterogeneous Data Sources

The ETL process (Extract, Transform, and Load) is critical to denormalize data for easy input in visual analysis tools. Unfortunately, this ETL process requires extensive human effort and computation to complete, often spanning months or years before in-depth analysis can be performed. In this paper, we introduce Klareco, a visualization architecture that foregoes the ETL process allowing quick access to multiple data sources. The architecture uses an indexing engine for accessing data with multiple schema. A series of small data analysis microservices add intelligence to the architecture. Finally, visualizations are designed to display and explore the data itself, as well as the structure of the data, facilitating discovery. This combination of features enables rapid prototyping of visualizations for a variety of data types, formats, and schema. We demonstrate an early version of the architecture using a case study in the domain of oil and gas exploration and to optimize production.

Paul Rosen, Alan Morris, Gene Payne, Bill Keach, Ian Walton, Bryony Richards-McClung, John McLennan, Randy Polson, Raymond Levey, Terry Ring, and others. Klareco: An Indexing-Based Architecture For Interactive Visualization Of Heterogeneous Data Sources. Workshop on Data Systems for Interactive Analysis (DSIA) at IEEE VIS, 2015.

@inproceedings{rosen2015klareco,
  title = {Klareco: An Indexing-based Architecture for Interactive Visualization of
    Heterogeneous Data Sources},
  author = {Rosen, Paul and Morris, Alan and Payne, Gene and Keach, Bill and Walton, Ian
    and Richards-McClung, Bryony and McLennan, John and Polson, Randy and Levey, Raymond and
    Ring, Terry and others, },
  booktitle = {Workshop on Data Systems for Interactive Analysis (DSIA) at IEEE VIS},
  year = {2015},
  abstract = {The ETL process (Extract, Transform, and Load) is critical to denormalize
    data for easy input in visual analysis tools. Unfortunately, this ETL process requires
    extensive human effort and computation to complete, often spanning months or years
    before in-depth analysis can be performed. In this paper, we introduce Klareco, a
    visualization architecture that foregoes the ETL process allowing quick access to
    multiple data sources. The architecture uses an indexing engine for accessing data with
    multiple schema. A series of small data analysis microservices add intelligence to the
    architecture. Finally, visualizations are designed to display and explore the data
    itself, as well as the structure of the data, facilitating discovery. This combination
    of features enables rapid prototyping of visualizations for a variety of data types,
    formats, and schema. We demonstrate an early version of the architecture using a case
    study in the domain of oil and gas exploration and to optimize production.}
}