The Silk Link Discovery Framework is a tool for discovering relationships between data items within different Linked Data sources.
Data publishers can use Silk to set RDF links from their data sources to other data sources on the Web.
The official homepage of Silk can be found on http://silk.wbsg.de
The Web of Data is built upon two simple ideas: First, to employ the RDF data model to publish structured data on the Web. Second, to set explicit RDF links between data items within different data sources. Background information about the Web of Data is found at the wiki pages of the W3C Linking Open Data community effort, in the overview article Linked Data – The Story So Far and in the tutorial on How to publish Linked Data on the Web.
The Silk Link Discovery Framework supports data publishers in accomplishing the second task. Using the declarative Silk Link Specification Language (Silk-LSL), developers can specify which types of RDF links should be discovered between data sources as well as which conditions data items must fulfill in order to be interlinked. These linkage rules may combine various similarity metrics and can take the graph around a data item into account, which is addressed using an RDF path language. Silk accesses the data sources that should be interlinked via the SPARQL protocol and can thus be used against local as well as remote SPARQL endpoints.
Silk can be used through the Silk Workbench web application or on the command line.
All variants are based on the Silk Link Discovery Engine which offers the following features:
- Flexible, declarative language for specifying linkage rules
- Support of RDF link generation (owl:sameAs links as well as other types)
- Employment in distributed environments (by accessing local and remote SPARQL endpoints)
- Usable in situations where terms from different vocabularies are mixed and where no consistent RDFS or OWL schemata exist
- Scalability and high performance through efficient data handling (speedup factor of 20 compared to Silk 0.2):
- Reduction of network load by caching and reusing of SPARQL result sets
- Multi-threaded computation of the data item comparisons (Over 1 billion comparisons per hour on a Core i7, 4GB RAM)
- Optional blocking of data items
Silk Workbench is a web application which guides the user through the process of interlinking different data sources.
Silk Workbench offers the following features:
- It enables the user to manage different sets of data sources and linking tasks.
- It offers a graphical editor which enables the user to easily create and edit link specifications.
- As finding a good linking heuristics is usually an iterative process, the Silk Workbench makes it possible for the user to quickly evaluate the links which are generated by the current link specification.
- It allows the user to create and edit a set of reference links used to evaluate the current link specification.
Silk is provided in three different variants which address different use cases:
Silk Single Machine is used to generate RDF links on a single machine. The datasets that should be interlinked can either reside on the same machine or on remote machines which are accessed via the SPARQL protocol. Silk Single Machine provides multithreading and caching. In addition, the performance can be further enhanced using an optional blocking feature.
Silk MapReduce is used to generate RDF links between data sets using a cluster of multiple machines. Silk MapReduce is based on Hadoop and can for instance be run on Amazon Elastic MapReduce. Silk MapReduce enables Silk to scale out to very big datasets by distributing the link generation to multiple machines.
Silk Server can be used as an identity resolution component within applications that consume Linked Data from the Web. Silk Server provides an HTTP API for matching instances from an incoming stream of RDF data while keeping track of known entities. It can be used for instance together with a Linked Data crawler to populate a local duplicate-free cache with data from the Web.
Support and feedback
For questions and feedback please use the Silk Google Group.
All stable releases can be downloaded.
The framework can be used under the terms of the Apache Software License.
The latest source code is available in the Git repository.
|| Various improvements and bugfixes.
|| Added support for active learning of linkage rules.
|| Various improvements to the Workbench.
|| Added support for learning linkage rules.
|| SPARQL/Update output.
Dump file input.
Many improvements to the Silk Workbench
|| Introduces per-comparison thresholds.
New data transformations and distance measures including token-based distance measures.
Improved Silk Workbench.
|| Includes the new Silk Workbench, a web application which guides the user through the process of interlinking different data sources.
|| Improved loading perfomance: Multiple parallel SPARQL queries are executed, while their results are merged on the fly.
Improved matching performance: New blocking method offers greatly improved performance.
Improved overall performance: Matching tasks are now executed concurrently to loading data instead of waiting for the complete data set to be loaded.
|| Added Silk MapReduce
|| Added Silk Server
Added a geographical distance metric by Konrad Höffner (MOLE subgroup of Research Group AKSW, University of Leipzig)
|| Reimplementation of the Silk framework in Scala. Improved scalability and performance.
Prematching replaced by a more transparent blocking.
Configuration is checked for consistency prior to link generation.
Support of the OAEI Alignment format. (Anja and Robert)
|| Added prematching of data items (Julius).
The Silk 0.2 language specification is still available and Silk 0.2 framework can be downloaded from GoogleCode.
|| Initial Release of the Python version of the Silk framework (Julius and Chris)