Recruiting NLP-crowdsourcing-semantic-web postdoc or staff scientist

Our laboratory at the The Scripps Research Institute in beautiful San Diego, California is recruiting a talented individual to help us use crowdsourcing to push the boundaries of biomedical information extraction and its applications.   We are looking for someone with experience in natural language processing (statistical or linguistic), machine learning, and knowledge representation.  This person would work to integrate efforts across several related projects.  

Ongoing and nascent projects include:

• Collaboration with the NASA tournament laboratory at Harvard University to use TopCoder innovation contests to improve the BANNER named entity recognition tool using data collected with Amazon Mechanical Turk.
• Collaboration with the Biosemantics group at Leiden University on applications of PubMed-scale knowledge discovery for human genetic diseases. 
• Collaboration with UCSD and Stanford to implement an open-access, biomedically-focused installation of the DeepDive knowledge base extraction system.  
• Collaboration with the BeFree team from the Integrative Biomedical Informatics Group of the Institut Hospital del Mar d’Investigacions Mèdiques (IMIM) to use the Crowdflower crowdsourcing platform to generate ground truth corpora for training a relation extraction system to be entered in the 2015 BioCreative challenge.  
• The development of the knowledge.bio application for browsing networks of extracted knowledge and gathering user feedback for improving the underlying knowledge base.  
• Ongoing development of the citizen science application Mark2Cure with an emphasis on its integration with all of the projects listed above.  

Sound like fun? Ready to jump in?

Contact Andrew Su and or Benjamin Good for more information.

p.s. We have other openings in related areas!

Gene Wiki SPARQL endpoint

Thanks to Leyla and Alex Garcia-Castro from UniProt and Florida State University respectively, we now have access to a SPARQL endpoint for the data in the Gene Wiki.  Access it live here:
http://virtuoso.idiginfo.org/sparql
(update on 4-28-12 , that is down and a live one is currently available at
http://199.102.237.69:8890/sparql
)

Here is one query that you might like to try that finds gene-disease links that we have mined from the text:

PREFIX wiki: <http://genewikiplus.org/wiki/Special:URIResolver/>

PREFIX property: <http://genewikiplus.org/wiki/Special:URIResolver/Property-3A> 

SELECT ?gene ?disease ?gene_name ?disease_name ?doid
WHERE {
 ?gene property:Is_associated_with_disease ?disease .
 ?gene property:HasSNP ?snp .
 ?snp property:Is_associated_with_disease ?disease .
 ?gene rdfs:label ?gene_name .
 ?disease rdfs:label ?disease_name .
 ?disease rdf:type ?disease_cat .
 ?disease_cat property:HasDOID ?doid .
 ?gene rdf:type wiki:Category-3AHuman_proteins .
}

How it works in brief

1. Articles from the Gene Wiki and from SNPedia are transferred to genewikiplus.org
2. As they go in, they are converted into a semi-structured form that enables queries in semantic media wiki.
3. We dump the entire thing out as one giant RDF file.
4. Leyla loads the RDF into their Virtuoso server (and performs some enhancements such as linking directly to UniProt RDF).
5. and wa la!

(More details about the generation of the genewiki+ are available in this soon-to-be-published paper about the SNPedia mashup and this paper about Semantic Wiki Links in Wikipedia.)

Cool next steps

The RDF has OWL:sameAs links between all the Gene Wiki entries and their RDF equivalents in DBpedia and in UniProt's RDF representation.  It should be possible to explore connections that span these three (four including SNPedia) resources using Linked Data technologies like Virtuoso's Sponger.

Go forth! Play with our data!

Semantic Wiki Links article published

A while back, I posted an entry here called "Stepping towards a Semantic Wikipedia" that quickly became the most-visited post on this blog.  Today, after several months of waiting, I discovered by accident that the official peer-reviewed article on the same subject is now available (the editors never bothered to tell us that it came out).  The article elaborates on a pattern that allows Wikipedia users to write content directly into the semantic Web.  The article is called "Building a biomedical semantic network in Wikipedia with Semantic Wiki Links" and is now available at DATABASE: The Journal of Biological Databases and Curation.  Here is the abstract:

"Wikipedia is increasingly used as a platform for collaborative data curation, but its current technical implementation has significant limitations that hinder its use in biocuration applications. Specifically, while editors can easily link between two articles in Wikipedia to indicate a relationship, there is no way to indicate the nature of that relationship in a way that is computationally accessible to the system or to external developers. For example, in addition to noting a relationship between a gene and a disease, it would be useful to differentiate the cases where genetic mutation or altered expression causes the disease. Here, we introduce a straightforward method that allows Wikipedia editors to embed computable semantic relations directly in the context of current Wikipedia articles. In addition, we demonstrate two novel applications enabled by the presence of these new relationships. The first is a dynamically generated information box that can be rendered on all semantically enhanced Wikipedia articles. The second is a prototype gene annotation system that draws its content from the gene-centric articles on Wikipedia and exposes the new semantic relationships to enable previously impossible, user-defined queries."

I will also be presenting a poster about this and related work at the upcoming Biocuration meeting in Washington DC.  Stop by for a chat!

Semantic Publishing workshop at ESWC in Greece

I'm helping to organize this exciting event, please consider submitting a manuscript and or attending.  From the official call for papers:

http://sepublica.mywikipaper.org/SePublica2012 an ESWC2012 Workshop.  May 27-31, Heraklion, Greece.
At Sepublica we want to explore the future of scholarly communication and scientific publishing. As we are going through a transition between print media and Web media, Sepublica aims to provide researchers with a venue in which this future can be shaped. Consider research publications: Data sets and code are essential elements of data intensive research, but these are absent when the research is recorded and preserved by way of a scholarly journal article. Or consider news reports: Governments increasingly make public sector information available on the Web, and reporters use it, but news reports very rarely contain fine-grained links to such data sources.  At Sepublica we will discuss and present new ways of publishing, sharing, linking, and analyzing such scientific resources as well as reasoning over the data to discover new links  and scientific insights. 

Workshop Format 

We are planning to have a full day workshop with two main sessions. During the first part of the workshop accepted papers will be presented; the second part of the workshop will address by means of focus groups two main questions, namely “what do we want the future of scholarly communication to be?”  and “how could data be preserved and delivered in an interactive manner over scholarly communications?”. These focus groups will be followed by a panel discussion. As an outcome of these activities we will have a communique that will be the editorial for the workshop proceedings,

Dates 

* workshop papers submission deadline: Feb 29
* workshop papers acceptance notification: April 1
* workshop papers camera ready: April 15 

Submission

 https://www.easychair.org/conferences/?conf=sepublica2012

Issues to be addressed

• Representation:
  • Formal representations of scientific data; ontologies for scientific information
  • What ontologies do we need for representing structural elements in a document?
  • How can we capture the semantics of rhetorical structures in scholarly communication, and of  hypotheses and scientific evidence?
  • Integration of quantitative and qualitative scientific information
  • How could RDF(a) and ontologies be used to represent the knowledge encoded in scientific documents and in general-interest media publications?
  • Connecting scientific publications with underlying research data sets

• Technological Foundations:
  • Ontology-based visualization of scientific data

• • Provenance, quality, privacy and trust of scientific information
  • Linked Data for dissemination and archiving of research results, for collaboration and research networks, and for research assessment

• • How could we realize a paper with an API?  How could we have a paper as a database, as a knowledge base?
  • How is the paper an interface, gateway, to the web of data? How could such and interface be delivered in a contextual manner?

Applications and Use Cases:

• Case studies on linked science, i.e., astronomy, biology, environmental and socio-economic impacts of global warming, statistics, environmental monitoring, cultural heritage, etc.
• Barriers to the acceptance of linked science solutions and strategies to address these
• Legal, ethical and economic aspects of Linked Data in science

Stepping towards a Semantic Wikipedia

(Update, check out our publication in Database for a full-length, peer-reviewed version of this article.)

It is now possible to specify the nature of the relationships between things described by Wikipedia articles directly in the context of the article.  The image below is a screenshot taken a few moments ago of the Phospholamban article on Wikipedia (with excited arrows added).  The infobox at the top right is dynamically generated from semantic markup in the article using a Wikipedia user script written by my colleague Sal and accessible from his user page. 

Semantic markup now live in Wikipedia

How it works

Wikilinks in the article have been annotated with the kind of relationship that they indicate using the Semantic Wiki Link (SWL) template. The template allows any Wikipedia editor (including you!) to specify the type of connection that exists between the article where the link is being placed and the target of the link. This information is encoded following the microformat pattern. Essentially, we encode the meaning of the links in class attributes that wrap the link.

This works as follows:

1. Editor inserts a SWL into a Wikipedia article with this syntax:
• {{SWL | target=protein kinase S | label=PKA | type =substrate_for}}
• This means "the concept where you see this link is related to protein kinase A (labeled PKA) with the relationship type "substrate for". So, in the example above, it says: "Phospholamban is a substrate for PKA".
• The {{}} denotes a Wikipedia template. Templates can take parameters (here parameters are separated by |'s) and use them to produce new WikiText dynamically which, in turn, is rendered as HTML when a page is loaded.
2. When the page is rendered, the template generates the following semi-semantic HTML markup (with some formatting omitted for clarity):

• 
  
3. Programs, like the script that generated that infobox and added the green highlighting, can look for the SWL class attribute can then extract the meaning of the SWL links based on the class of its first child element - here "substrate_for".
4. In addition, when the template is processed it adds a category to the article it is placed on that corresponds to the relationship type. (See for example, the category for substrate.) This category provides a logical grouping (e.g. all things that serve as a biochemical substrate) but, perhaps more importantly, it provides a place to record the meaning of the relationship. This meaning can be defined as text, but can also be defined through reference to external sources such as ontologies on the semantic web.

Why its awesome

This pattern makes it possible for the vast number of Wikipedia users to simply and easily contribute machine readable content to the Web. This enormous user community collaboratively created the world's largest encyclopedia and one of the most valuable websites on the planet. Who better to help build the semantic Web? While, technically, the microformat-like implementation leaves much to be desired in terms of its robustness and its precision, it is a solution that can work. This is demonstrated by the success of projects like Google's recent recipe search that are based entirely on simple microformats.

How you can help

This is a new idea that not everyone in Wikipedia will be thrilled to see. They will claim that the SWLs will clutter the markup and will not provide enough value to make it worth it because Wikipedia itself does not support semantic links. You can help by:

• Using the template to enhance articles.
• Writing code that makes use of the added meaning such as user scripts, aggregators, or scripts that import the relationships into other structured repositories like FreeBase or DBpedia.
• Helping define the nature of the semantic links (at their associated category pages) and mapping them to properties defined in ontologies.
• Discussing (and voting for) the idea on the various 'talk pages' on Wikipedia.

Why its awesome again

Did I mention that this pattern makes it possible for the vast number of Wikipedia users to simply and easily contribute machine readable content to the Web? Thats pretty cool if you think about it...

Update: To make the user script work for you so you can see the infobox, do this:

• Create a Wikipedia user account if you don't have one already
• Go to/create your user page. (e.g. my user name there is i9606 and my user page is located at http://en.wikipedia.org/wiki/User:I9606 )
• Edit your user page add this to it - 

[[/common.js]]

• Visit your new common.js add this to it - 

importScript('User:Sal9000/SWLinfobox.js');

• When that is saved, you should be all set. Now go visit an enhanced page like Phospholamban and look for the green box at the upper right corner.

The script will run whenever you access a Wikipedia page while you are logged in to your account. Its a lot like GreaseMonkey, (which I've had some fun with in the past) but its not tied to your browser and will only work on Wikipedia. If enough people like a user script, it can be added to the default set of Wikipedia user preferences.. which would be pretty cool ;). The best part? You (or a programmer friend of yours) can write your own script and make it do whatever you want with the data!

Semantic Web Service Course in Vancouver

My old friend and PhD advisor Mark Wilkinson is organizing an upcoming short course on semantic web services. Aside from a basic intro to the relevant pieces of the semantic web, you will become deeply ingrained in SADI.  See below for details.

Training Courses in Web Publishing - Scientific Data and Services in Vancouver, BC, Canada.

Updated information!!

 Registration                              Workshop
Until October 21st, 2011              October 22nd-23rd, 2011

"The Semantic Web is emerging as a new and more powerful standard for data and knowledge representation compared to the traditional Web. In this very unique, hands-on workshop you will learn how to use the Semantic Automated Discovery and Integration (SADI) framework to publish your data and analytical tools on the Semantic Web to make them easily accessible, and easier to integrate with other data and tools. Space is limited: don't get left behind in the Semantic paradigm shift! Register today at http://sadiframework.org/training/WS2/index.html."

The course provides theory and practical skills in the semantic technologies that underpin the development of semantic web services.The training course is divided into the following modules:

• Semantic Web and DL Reasoning

• SADI Semantic Web Services

The course provides an excellent learning opportunity and serves as a venue for the exchange of ideas among a highly interdisciplinary group of scientists. The intended audience for the course are:

• Industry pioneers in workflow, grid and semantic technologies;

• Web developers and researchers seeking to ensure the adoption of their technologies as part of a growing ecosystem of easily discoverable and accessible data and services;

• Grid computing architects and developers interested in building globally interoperable systems;

• Workflow system developers coordinating computational and data resources across the web using semantics;

• Researchers involved in web-based knowledge discovery

  The cost per participant is:
  Regular  Registration (375 CAN$)
  Student  Registration (325 CAN$)On site Registration (500 CAN$)
  

We ask that you broadcast this information widely to your colleagues. Our flyer can be found at http://sadiframework.org/training/WS2/WS2_flyer.pdf. Please do not hesitate to contact us at spaixao@unb.ca  should you have any question.

CfP Semantic Publishing

Minoan rhyton from Crete!

I'm on the program committee for this conference workshop so clearly you should submit something (and its in Crete!).  See the call for papers below:

-----------------------------------------------------------------------------------------------------------------

1st International Workshop on Semantic Publication (SePublica 2011)

http://sepublica.mywikipaper.org

at the 8th Extended Semantic Web Conference (ESWC 2011)

http://www.eswc2011.org

May 29th or 30th, Hersonissos, Crete, Greece

Keynote by Steve Pettifer, Manchester University, UK.

“Utopia Documents and The Semantic Biochemical Journal experiment”

SUBMISSION DEADLINE February 28

The MISSION of the SePublica workshop is to bring together researchers

and practitioners dealing with different aspects of Semantic

Technologies in the Publishing Industry. How is the Semantic Web

impacting the publishing industry? How is our experience of

publications changing because of Semantic Web technologies being

applied to the publishing industry?

The CHALLENGE of the Semantic Web is to allow the Web to move from a

dissemination platform to an interactive platform for networked

information. The Semantic Web promises to “fundamentally change our

experience of the Web”.

In spite of improvements in the distribution, accessibility and

retrieval of information, little has changed in the publishing

industry so far. The Web has succeeded as a dissemination platform for

scientific and non-scientific papers, news, and communication in

general; however, most of that information remains locked up in

discrete documents, which are poorly interconnected to one another and

to the Web.

The connectivity tissues provided by RDF technology and the Social Web

have barely made an impact on scientific communication nor on ebook

publishing, neither on the format of publications, nor on repositories

and digital libraries. The worst problem is in accessing and reusing

the computable data which the literature represents and describes.

• Consider research publications: Data sets and code are essential

elements of data intensive research, but these are absent when the

research is recorded and preserved in perpetuity by way of a scholarly

journal article.

• Or consider news reports: Governments increasingly make public

sector information available on the Web, and reporters use it, but

news reports very rarely contain fine-grained links to such data

sources.

QUESTIONS AND TOPICS OF INTEREST

• What does a network of truly interconnected papers look like?

How could interoperability across documents be enabled?

• How could concept-centric social networks emerge?

• Are blogs and wikis new means for scholarly communication?

• What lessons can be learned from humanities and social science publishers

(i.e. going beyond scientific publishing towards scholarly publishing)?

• How could we move beyond the PDF?

How can we embed and link semantics in EPUB and other e-book formats?

• How are digital libraries related to semantic e-science?

What is the relationship between a paper and its digital library?

• How could we realize a paper with an API?

How could we have a paper as a database, as a knowledge base?

• How is the paper an interface, gateway, to the web of data?

How could such and interface be delivered in a contextual manner?

• How could RDF(a) and ontologies be used to represent the knowledge encoded

in scientific documents and in general-interest media publications?

• What ontologies do we need for representing structural elements in a

document?

• How can we capture the semantics of rhetorical structures in

scholarly communication, and of  hypotheses and scientific evidence?

AUDIENCE

• researchers from diverse backgrounds such as argumentative

structures, scholarly communication, multi-modality in publications,

digital libraries, semantics in publications, and ontology

engineers.

• practitioners active in the publishing industry, repositories of

experimental information and document standards.

IMPORTANT DATES

Paper/Demo Submission Deadline: February 28, 23:59 Hawaii Time

Acceptance Notification: April 1

Camera Ready Version: April 15

SePublica Workshop: May 29 or May 30 (to be announced)

SUBMISSION AND PROCEEDINGS

Research papers are limited to 12 pages and position papers to 5

pages. For system descriptions, a 5 page paper should be

submitted. All papers and system descriptions should be formatted

according to the LNCS format

http://www.springer.com/computer/lncs?SGWID=0-164-6-793341-0

We encourage the submission of semantic documents. LaTeX documents in

the LNCS format can, e.g., be annotated using SALT

(http://salt.semanticauthoring.org) or sTeX

(http://trac.kwarc.info/sTeX/). We also invite submissions in

XHTML+RDFa or in the format or YOUR semantic publishing tool.

However, to ensure a fair review procedure, authors must additionally

export them to PDF.  For submissions that are not in the LNCS PDF

format, 400 words count as one page. Submissions that exceed the page

limit will be rejected without review.

Depending on the number and quality of submissions, authors might

be invited to present their papers during a poster session.

Please submit your paper via EasyChair at

http://www.easychair.org/conferences/?conf=sepublica2011

The author list does not need to be anonymized, as we do not have a

double-blind review process in place.

Submissions will be peer reviewed by three independent

reviewers. Accepted papers have to be presented at the workshop

(requires registering for the ESWC conference and the workshop) and

will be included in the workshop proceedings that are published online

at CEUR-WS.

PROGRAM COMMITTEE

• Robert Stevens, Manchester University, UK

• Benjamin Good, GNF, USA

• Michael Kohlhase, Jacobs University, Germany

• Oscar Corcho, Politecnica de Madrid, Spain

• Steve Pettifer, Manchester University, UK

• Jodi Schneider, DERI, NUI Galway, Ireland

• Sebastian Kruk, knowledgehives.com, Poland

• Henrik Eriksson,  Linköping University, Sweden

• Dagobert Soergel, University of Maryland, USA

• Tim Clark, Harvard Medical School, USA

• Paolo Ciccarese, Harvard Medical School, USA

ORGANIZING COMMITTEE

• Alexander García Castro, University of Bremen, Germany

• Christoph Lange, Jacobs University Bremen, Germany

• Anita de Waard, Elsevier, USA/Netherlands

• Evan Sandhaus, New York Times, USA

QUESTIONS? → sepublica@googlegroups.com

Shakey on the Web

Mike Bergman has a nice post about the lack of appropriate means for representing ambiguity on the Web that is worth a look.

It reminded me again of Shakey the robot - its promise and its ultimate failure. Shakey was one of the first 'autonomous' robots; capable of making its own decisions to guide its actions based on data from its sensors. In its heyday in the 60's and 70's it stirred up Jetsonian imaginations of robot vacuum cleaners and so forth but in the end, finally failed to deliver anything of the kind. Shakey's mind was trapped in a strict, orderly world of logical rules. Much like the character Brooks Hatlen from the Shawshank Redemption, when faced with the uncertain reality of the our world he simply couldn't cope. (Watch him shake in trepidation!)

The robot minds that did end up making it into our vacuum cleaners are based on very different kinds of 'minds'. In a movement initiated by Rod Brooks in the late 80's, now personified at home by the iRobot vaccum cleaners, in the academy by the field of 'probabilistic robotics' and on the podium by Stanley, the rules and central planning centers of good old fashioned AI are gone. In their place, we find statistical models that - like the world they were built to deal with and perhaps a bit more like human minds - constantly change to reflect the uncertainty of reality.

I fear that our current approach to "reasoning" on the semantic web bears much more in common with old Shakey than it does with Stanley.

Presenting at San Diego Semantic Web Meetup

In a few weeks, I will be giving a seminar for the San Diego Semantic Web Meetup group. Its going to cover a fair amount of the same material we covered at the NCBO webinar, but you should come anyway because a) I have new data and b) I'm going to be there live (so you can bring your rotten tomatoes!).

** Update, I am now excited to be presenting with Denny Vandrecic, an expert on ontology evaluation and one of the founders of the Semantic Media Wiki.

Peter Mika at SD Semantic Web Meetup tomorrow

For anyone in San Diego with an interest in the Semantic Web, tomorrow's meetup is a must. The presenter for the day will be Peter Mika from Yahoo! Research in Barcelona, stopping in on his way to SemTech. Peter gained early career fame for investigations into the relationships between social networks and the semantic web such as the award winning "Ontologies are us: a unified model of social networks and semantics". Abstract from the Meetup site below, hope to see you there.

"While current search techniques aim at ever more sophisticated methods for searching over hypertext, the Semantic Web promises to break boundaries in search by transforming the content itself into a form that is more easily processable by machines.

In this talk, we will discuss some of the possible technologies for annotating content for machine processing and showcase some of the ways that semantic annotations can improve the search experience for users. In the first part of the talk, we will describe existing and upcoming formats for embedding metadata inside web content, including microformats, RDFa, Facebook's Open Graph format and HTML5's microdata. We illustrate these formats with practical examples, show some of the tools that can help authoring and give guidance on best practices and point out some of the pitfalls, drawing on our experience from working with large publishers. In the second part of the talk, we will discuss research in semantic search and demonstrate its applications in Semantic Web search engines and web search engines with semantic extensions, including Yahoo's Search Monkey and Google's Rich Snippets."

Jerzy Lewak at SDSW Meetup

Image via Wikipedia

I just got back from another interesting San Diego Semantic Web Meetup, here are my notes before I forget.

The presenter this evening was Jerzy Lewak, a former theoretical physicist, a professor emeritus at UCSD, and cofounder of (at least) Nisus Software and SpeedTrack Inc.. (Once again, I continue to be impressed at the level of the speakers that are recruited for these events!). Jerzy presented the history and current applications of a novel human interface for databases that he calls GIA (Guided Information Access) that is made possible by the underlying TIE (Technology for Information Engineering) framework.

Jerzy began his talk as a true computer scientist by defining 'The Problem' that originally inspired this work (back in 1991). At that time email was just starting to pick up steam and already the task of finding old emails was becoming unmanageable. So, he set out to find a better way. (Always nice to be working on a solving a problem for yourself.) His search for a solution took a decidedly familiar path:

1. Hmm, why don't I set up some nice hierarchies of concepts to place my emails into so it will be easier to find them later?
2. Darn.. that really isn't working out very well. The more data I get, the harder it is to organize and I keep running into the problem that almost every single item in my collection might be placed under more than one category. Perhaps a physical filing cabinet is actually a terrible thing to base a completely virtual information storage and retrieval system on... (Though he didn't bring it up, he was describing exactly the same thing that Clay Shirky got so excited about in the 'Ontology is Overrated' essay in 2005 that - in turn - got me all excited in 2006, except of course Jerzy was thinking in 1991. And, as pointed out to me by my LIS friend Joe, this basic problem and the following conclusion were pretty well fleshed out by Ranganathan in the 1930's...).
3. After experimenting with plain content search (a la Google) he arrived at faceted classification as the most powerful and flexible way to describe and access data.

So far so good, I am paying attention.

Now the problem arises that many combinations of facets actually produce zero results. As he noted, just 200 facets (he uses the word 'selectors') is enough to uniquely describe every particle in the universe. This became the real problem. The breakthrough that got him going and has led to SpeedTrack and everything else he presented was the idea of dynamically limiting the potential facets based on those that are already selected. In the interfaces he demonstrated, he would:

1. Choose some database field like 'last name' and type in a name like 'Smith'
2. Show two things - one, the number of results went down and two, the number of possible values for the other fields (e.g. 'first name', 'height', 'date', etc.) would immediately be constrained to only show values for objects linked to Smiths in the database.
   

His interfaces might be described as an advanced multi-parameter type-ahead. They work by guiding the user (GIA) in the creation of potentially very complex queries that are guaranteed to return results. This is achieved by dynamically exposing the underlying indexes that drive boolean queries.

The system works on both structured and unstructured data (he showed a quick example of newspaper articles) but requires fairly heavy manual labor to get it started. Overall it looked like it would be useful and fun to use and I can imagine many potential directions they could take it.

My only complaint for the talk was that there was absolutely zero Web in it - no mention of any native ability to consume or produce RDF, no mention of OWL, no discussion of scaling possibilities, and the proverbial elephant in the room of large-scale data access was left more or less untouched. I guess that might be one sign of a good talk - I got interested and it left me thirsting for more..

What would you do with a semantic pubmed?

If all of the concepts buried in the text in PubMed were dug out, stitched together into a semantic network and placed into a system that could rapidly provide answers to database-style (e.g. sparql) queries over that graph what would you do with it?

I was recently posed this question and have been scratching my head to come up with a very good answer. I can't seem to get past the first thought that popped into my head. My initial idea was to build an interface that would allow any user to gain answers to this fundamental question

"how is concept A related to concept B".

As I saw demonstrated a few weeks ago, the technology exists to execute such queries but I personally have never seen a web interface that would allow untrained end users to explore very large knowledge bases in that fashion. I imagine something as simple as two autocomplete boxes for the input and then a variety of presentation styles for the output (graphs, text, etc.).

Any other ideas ? I might get a chance to build something like this soon.

-Update (link to small related friendfeed thread)

Jans Aasman From Franz Inc. Presents at San Diego Semantic Web Meetup

Yesterday, about 20 people gave up 4 hours of sunny Saturday afternoon to gather in Carlsbad to listen to Jans Aasman present a tutorial on AllegroGraph - what Franz Inc. is touting as "Web 3.0's Database".  The attendees, some of whom drove all the way down from Santa Barbara, included a number of people interested in semantic search and search engine optimization.  There were also software developers, CEOs of software development companies, bioinformaticians, an orthopedic surgeon, business development specialists, and entrepeneurs.  A challenging crowd to give a tutorial too that all would find useful.

Since I ended up having to take my notes on my iphone and the presentation lasted nearly 4 hours, I'm only going to hit a couple of the things that stood out for me here.  For more info. on the products presented, the Franz Inc. website has extensive information - and free downloads!  I also hope to find a link to the slides at some point.

First of all, wow - well done organizer!  Its not everyday that you get to hear a presentation by the CEO of one of the oldest (is it the oldest?) living companies involved in artificial intelligence.  (Yeah I said it.. thats what we used to call this stuff.)  Franz has apparently weathered the storm and now that semantics is coming back to life under a new name - has come out doing very well with clients including: Lilly, GlaxoSmithKline, Adobe, Raytheon, Kodak, Boeing, Cisco, Mayo clinic, Novartis, and many others.   These clients are paying for the pro versions of tools like AllegroGraph and Gruff as well as consulting services related to the use of these tools to solve specific problems.  Jans began his presentation with a really rapid explanation of why these companies are spending their money like this rather than on continued development with the encumbent technology - relational databases.

Jans suggested that triple stores like AllegroGraph are more useful then relational databases when any of the following constraints are met:

1. You have many classes of complex objects
2. The properties/definitions of these classes change frequently
3. You want to work with rules/reasoning
4. You have a big graph in your data that you want to analyze 

Of those he hit on #2 the most often.  Basically, if you can perfectly conceive of the entire world that you need to model at the time you create your database schema you are probably going to be better off using the more mature rdbms technology.  If not - and this is the likely case for more projects then your database architect would have you believe - the triple tech. will make change much easier.

After that brief, high-level motivator he dived into a fairly extensive series of demos/examples.  While the subject matter varied from news stories to clinical trials to digital photo websites, there were two consistent themes that I found interesting.

1. Every example he gave involved a named entity extraction step at one point or another.  This indicates that, while there is a growing amount of structured data out there, there is still way too  much information held in text to ignore when doing any real analysis.  So..  you are going to need to get friendly with some people that do natural language processing - perhaps like this company Alchemy.
2. Most of the demos were conducted in Gruff, the visual interface to AllegroGraph.  In Gruff, he repeatedly showed off the very cool trick of selecting two, apparently unrelated nodes in the graph (say a two different drugs) and asking for connections between them.  AllegroGraph responded to this request ~instantaneously with relevant and useful results.  Very impressive technically (this was a graph with millions of triples and the system was running on his laptop) and I think very useful for discovery.

Wish I could write more but its the weekend after all..  For more information on this stuff, go to Franz inc., grab yourself a free download or two, and try it out yourself!  I'd be curious to hear how you find it.

Eric Little presents at San Diego Semantic Web Meetup

Just returned from my first Semantic Web Meetup in San Diego, here are some impressions:

In comparison to the Vancouver SW meetups that I have attended

1. Quite a different crowd. Larger group, zero t-shirts, generally older,
   stronger industry representation, heavily weighted towards biomedical
   folks - even an actual surgeon in attendance.
2. As with everything in Southern California, it involved a fairly long drive.
3. If I was a small fish in Vancouver, I'm a microbe here.

A few bullets from the (very good) talk

1. The presenter, Eric Little, did his PhD with Barry Smith (in formal biomedical ontology) and came from a background in philosophy. He is now the chief knowledge engineer at CTG - a fairly large (revenue in the 100's of millions) Information Technology and Consulting company.
2. He talked for a while about how ontologies should provide "actionable intelligence" and gave some nice stories about the benefits gained when links are formed across semantic groups - for example, they created a diagnosis ontology by linking the disease ontology to the LOINC (logical observations, identifiers, names, and codes). Kind of the same old story here, but good examples.
3. He then got to the meat of the talk (according to the abstract), which was about their MedMap application. He, correctly IMHO, pointed out that pretty much every attempt at an interface for dealing with large ontologies more or less sucks. No offense... He then proceeded to claim that their solution in the MedMap application was much better. The comparison to things like Protege and Concept Map builders was a bit unfair as the MedMap application is targeted towards end users rather than ontology engineers but, I can't say I disagree with the picture he painted.
4. The MedMap application is a very nifty widget-based approach built using Adobe Flex on the top and with combination of Top Quadrant products underneath (notably SPARQLMotion). It looks a lot like the iGoogle interface, with a user-customizable collection of what he called "analysis portlets". Each of the portlets has a dedicated task, but they all sit on top of the same massive knowledge base. The examples he gave were clinical - for example, he showed one scenario where patients were being automatically classified into different risk groups with a nice little graphical representation of the different classes; however, the technology and many of the interface modules are clearly fairly general purpose. He mentioned that they are in fact working in quite different domains such as the petroleum industry.
5. To get your own MedMap-like solution for your problem, starting rates are on the order of $300-400,000 for about a 3 month long push for a first usable product. They are after big fish like the health insurance industry.
6. His opinion was that the only companies that are likely to be interested in buying semantic IT solutions are those that a) have a lot of money, b) are more forward-thinking than average, and c) have already tried other approaches (e.g. relational databases) and watched them fail.

Personal notes/reactions

1. The sheer slickness of that Flex interface still has me all in a quiver.. It makes me want to delete everything we have been doing on my current attempt at a semantic application (I am now embarrassed to put in the link) and start over - or perhaps it is just time to concede that its impossible to build Ferraris like I what just saw without employing a staff of people that know what they are doing.
2. It was cool to reconnect with Barbara Starr, my former supervisor at SAIC - which was prior to my MSc, my foray into bioinformatics, and my PhD! Grey hairs...
3. It was also very cool that she is about to start working on an application of the GoodRelations ontology by Martin Hepp. I am proud to say that Martin sent me unsolicited positive feedback about my first research project in grad school several years ago. He concurred with my claim that ontology development/maintenance practices were too expensive given typical knowledge engineering practices and that techniques that tapped into the collective knowledge of the web (a la 2.0) were the way to go. (Note that this contention is in stark contrast to the ontologies of "reality" from Barry Smith.)

All in all a very good meeting and I am very much looking forward to the next one.

Heading to China

Image via Wikipedia

Tomorrow I am departing to attend the Asian Semantic Web Conference in Shanghai, China.  I'll be manning the demo for my former labmate's project CardioSHARE (which I played no real part in building).

Looking forward to getting caught up on the latest from the semantic web research community.  Out of the accepted papers, I am most looking forward to hearing "Merging and Ranking answers in the Semantic Web: The Wisdom of Crowds".

Hope to see you on the other side of the Great Firewall.

Birth and Rebirth

Its now been 4 weeks, one hour and about 51 minutes since my son was born.  Its also been about  6 months since I finished my PhD and about 6 1/2 years since I had a 'real' job.  Perhaps its time to get on with things and sort out what I'm really going to be doing with myself.  Here's an update on what I've been up to, opinions on my next steps would be welcome...

Having grown somewhat disillusioned with the academic world, I've spent the time since graduation working completely outside of it.  Working with my father and his partner I learned how to use the Google App Engine while building a website/database for their company TrueIDapps.  I've also been involved with the development of a semantic web based startup that is now going by the name FreeForm Information.  Sadly I have not done any work in the domain of bioinformatics for a long time now.

While my post-grad projects have provided some knowledge of fun new techniques (I love the Cloud like everyone else now) and I am beginning to get a glimmer of an understanding of the process of starting and running a business, I have as yet to make a single penny on either project.  Now, faced with the screaming, squirming reality of responsibility I am feeling the pressure to make some decisions about how I should be spending my - now much more limited - work time.

Do I :

1. Continue to try to keep myself involved with both FreeForm and TrueIDapps - in the hope that one of them will eventually pan out?
2. Focus my attention on TrueIDapps because I have two other fulltime coworkers versus one part-time partner, because it always feels good to support the family business and because it seems that it is close to making money?
3. Focus my attention on FreeForm because I find the project more interesting, more relevant to my past experience and more inline with the work I would be trying to do if I managed to find a real job?
4. Drop both projects and figure out a path towards a real* job?
5. Commit to being Dr. Daddy, buy some baby formula and get Dr. Mommy back to work ?
   

I have the benefit of my wife's savings and great family support so I could certainly last for a while before I hit the financial danger point but..  the pressure is mounting. 

Your insights are most welcome!

*real job = a job with a salary, paid vacation, an office, - and a boss who tells you what to do.

Semantic media retrieval service? please?

Here is an application of semantic Web technologies that I would like to have. Please make it for me so that I don't have to. When I finish writing this post,

1. I would like to press a button that said "Enhance?".
2. When I pressed the button, the application would read through the text and identify terms, phrases, or other conceptual nuggets that it 'understood'.
3. These concept nuggets would then be used to find stock / open access images (and videos, etc.)
4. Where a likely candidate set of images was identified, they would be displayed such that I could quickly choose which, if any, that I liked
5. When I agreed to keep one, it would be embedded in a reasonable location in the text and I would very rapidly go on with my life, but with the added joy of having authored a much more entertaining piece of online personal history.

This thought crept into my mind after reading through Joey de Villa's post about joining Microsoft which is shot full with entertaining media enhancements to the text - which likely took a non-insignificant amount of time for him or his team of personal assistants to put together.

Pictures are indeed worth many words, but how many $$$'s? Perhaps you might even be able to make money with such an app by using it to sneakily sell professional photos and other content.

While you are at it, could you please provide the same text-to-media service in a non-embedded application so that when I needed a clever portrayal of a concept like 'failure', 'success', or 'mass collaboration', for a presentation I could quickly look one up. I might even be willing to by it if the content was good and the price was reasonable -> in a world where I could almost certainly find what I needed by spending a little more of my own valuable time looking for it.

Freebase authors semantic web book

I just noticed this post on the freebase dev blog - announcing the release of a new book about "programming the semantic web" written by freebasers Toby Segaran, Colin Evans, and Jamie Taylor. I haven't picked it up yet, but based on the description there and in the amazon reviews, it looks like a(nother) nice explanation of the concepts involved in the semantic web as well as a set of practical programming examples based on the W3C standards (OWL, RDF, SPARQL, etc.).

I found it surprising that freebase was not mentioned anywhere in the brief description given in the post or on Amazon.

CWA at the YMCA

Somewhere high in the air between New York and Minneapolis, my first stop on my way home, I feel compelled to explain a few things to myself.  Why on Earth have I just spent the last several nights living in the YMCA in Flushing, New York?  Why did I decide to go on my first self-funded professional excursion at a time when I have no income and very little savings?  What did I hope to get and what did the trip deliver? 

The inspiration for this minor adventure was the inaugural meeting of the Concept Web Alliance (CWA) at the New York Hall of Science.  The mission statement of the CWA (written partly at this meeting) is as follows:

“To enable an open, collaborative environment to jointly address the challenges associated with high volume scholarly and professional data production, storage, interoperability, and analyses for knowledge discovery”

The idea is to form an alliance of like-minded researchers and science publishers interested in sharing knowledge in a computationally accessible fashion (i.e. not plain text and such that information from multiple sources can easily be integrated and interacted with).  The basic building block envisioned for these efforts is the ‘triple’ – a Concept-Relation-Concept structure.  (The word ‘triple’ and the interesting new verb ‘triplification’ - meaning to convert some non-triple-structure like text into a set of triples - were almost certainly the most commonly uttered words in the presentations at the meeting.) 

For those familiar with semantic Web standards such as RDF (a generic triple-based language for representing and sharing information) and OWL (a set of languages for representing knowledge in the form of ontologies) it is perhaps most interesting to consider what is not present in a CWA triple and what was not discussed at all in the public portions of the meeting.  The following words never came up ‘description logic’, ‘axiom’, ‘class’, ‘reality’.

The intended materialization of the Concept Web vision - at the moment - thus seems to be an open collection of informal (non logic-based) concept representations, identified by URIs retrievable on the Web, that can be linked together to form semantic networks.  This graph-structure could be queried (e.g. using SPARQL) for the ‘facts’ that it would contain where each such fact would be linked to extensive information about where it came from (who (or what algorithm) suggested it, when, and with what confidence).  Interestingly, this is very similar in its flexibility, lack of built-in reasoning, and its strong notion of provenance tracking to the Freebase model. 

While some of you who like to work with reasoners and OWL or who think that it is better to talk about ‘universals’ and ‘particulars’ than it is to talk about ‘concepts’ may find this lack of formality a little disappointing, I am growing more and more enthusiastic about it because fits the publish-then-filter nature of the Web perfectly.  We see again and again that once information is out there on the Web, its value increases tremendously.  (In fact, many very smart people seem to think that when there is enough text and other unstructured data online that is all we will really need to solve most of our information problems.) By providing a very low barrier for entry and then focusing computer science efforts on handling the noise, complexity, and the conflicts that will inevitably arise (the filter part), I think this triple-publishing approach has great potential to push research in a productive direction.  In particular, I think it will push people to spend more time working on other, more flexible modes of inference that don’t just die when a logical conflict is detected - all that squishy probability stuff that the semantic Web has managed to ignore for so long and that happens to be the stuff that makes almost all interesting AI-like technology work now.  Furthermore, the triple-focus absolutely does not stop groups that participate in the CWA from making use of approaches grounded in formal logics in their own development. 

While there may or may not be good reasons to take one philosophical stance over another when creating knowledge bases, the fact will always remain that there will be conflicts of opinion about this.  When dealing with a small group, it may be possible to convince or force acceptance of a particular world view, but it is not IMHO going to be possible to enforce something as arguable as the philosophy of the representation of the nature of being on the scale of the Web.  By focusing on the smallest possible units, the triples, and leaving the more precise formalizations and the philosophy out of the vision as much as possible, the CWA might make it possible for a diverse, interoperable ecology of knowledge bases to emerge and co-exist.  Ideally, those who wish to make use of, for example - description logic reasoning, should be able to benefit from the pool of URIs in the Concept Web if for nothing other than for the many multi-lingual labels and textual definitions that will be associated with each of them. 

It is still very early days for the CWA – probably far too early to really speculate too far about the consequences of its basic technological approach as even this approach is still very much up for debate.  Still..  I’m not sure exactly how to express this, but the meeting smelled good.  There were enough capable, powerful, enthusiastic people together in that room that seemed to have enough of a shared vision that I think it is very likely that something of that vision is likely to come to life.

So, was it worth it?  I think that it was in the end.  I got an early look at something that might provide solutions to many of the problems that I’ve spent the past several years of my life thinking about (the social construction of a biosemantic web).  I got to reconnect with old friends and make some new ones.  I had a chance to see New York for the first time (the scale of which blew my mind).  And, last but not least, it just might be the last such academic event I get to take part in.  Depending on the choices I make and the dictations of the wheels of fate I may be in the process of losing the privilege of working in the ivory tower.  If it was indeed my goodbye to the community of scholars, it was a good one.

So yes, it was a worthwhile trip and it remains an exciting time to be thinking about the Web - concept or otherwise.  (and the YMCA wasn’t really so bad in the end ;).

You can follow - and perhaps influence - the evolution of the Concept Web on their blog.

Dissertation now online

The full text of my dissertation, "Strategies for amassing, characterizing, and applying third-party metadata in bioinformatics", is now available via UBC's information repository.  It is "manuscript-based" so each of the chapters except the introduction and the conclusion can be read and understood independently.  (So there is really no reason for anyone ever to try to read the whole thing in its entirety.)  

Here is the abstract:

Bioinformatics resources on the Web are proliferating rapidly. For biomedical researchers, the vital data they contain is often difficult to locate and to integrate. The semantic Web initiative is an emerging collection of standards for sharing and integrating distributed information resources via the World Wide Web. In particular, these standards define languages for the provision of the metadata that facilitates both discovery and integration of distributed resources. This metadata takes the form of ontologies used to annotate information resources on the Web. Bioinformatics researchers are now considering how to apply these standards to enable a new generation of applications that will provide more effective ways to make use of increasingly diverse and distributed biological information. While the basic standards appear ready, the path to achieving the potential they entail is muddy. How are we to create all of the needed ontologies? How are we to use them to annotate increasingly large bodies of information? How are we to judge the quality of these ontologies and these proliferating annotations? As new metadata generating systems emerge on the Web, how are we to compare these to previous systems? The research conducted for this dissertation seeks new answers to these questions. Specifically, it investigates strategies for amassing, characterizing, and applying metadata (the substance of the semantic Web) in the context of bioinformatics. The strategies for amassing metadata orient around the design of systems that motivate and guide the actions of many individual, third-party contributors in the formation of collective metadata resources. The strategies for characterizing metadata focus on the derivation of fully automated protocols for evaluating and comparing ontologies and related metadata structures. New applications demonstrate how distributed information sources can be dynamically integrated to facilitate both information visualization and analysis. Brought together, these different lines of research converge towards the genesis of systems that will allow the biomedical research community to both create and maintain a semantic Web for the life sciences and to make use of the new capabilities for knowledge sharing and discovery that it will enable.

About the title..  I ended up using the generic "metadata" rather than something more specific because I needed a way to concisely capture things that range from Del.icio.us tags to classes in the Foundational Model of Anatomy.  "Metadata" seemed to do the job, but it remains a little vague and therefore unsatisfying.  Similarly, "third-party metadata" is broader than necessary.  I don't touch institutionally generated third-party metadata - just what I guess you would probably call "socially generated metadata".

Enjoy.