MQL Cookbook
From Freebase
This page contains a list of recipes for common needs using the MQL query language.
Feel free to add your own!
Get data as of a specific time
You can use the as_of_time attribute in the mql envelope to get data that was present at a specific time.
this query:
{
"q1" : {
"as_of_time" : "2007-01-09T22:00:56.0000Z",
"query" : [
{
"domain" : "/architecture",
"id" : null,
"return" : "count",
"timestamp" : null,
"type" : "/type/type"
}
]
}
}
returns 12 (there were 12 types in the /architecture domain last january).
{
"q1": {
"code": "/api/status/ok",
"result": [
12
]
},
"status": "200 OK",
"code": "/api/status/ok"
}
this query (no as_of_time attribute):
{
"q1" : {
"query" : [
{
"domain" : "/architecture",
"id" : null,
"return" : "count",
"timestamp" : null,
"type" : "/type/type"
}
]
}
}
returns the current number of types in the architecture domain:
{
"q1": {
"code": "/api/status/ok",
"result": [
25
]
},
"status": "200 OK",
"code": "/api/status/ok"
}
This example was extracted from a response on the developers@freebase mailing list.
One interesting use of this feature is to create a kind of trust authority application. Establish your own trusted view of topics by store the timestamp of a known good version. Then query to get the trusted version.
Read properties of a type
This brings up the general concept of "default_property"
If I ask for two property values with null and []:
{
"q":{
"query":{
"active_start":null,
"genre":[],
"name":"The Police",
"type":"/music/artist"
}
}
}
I get the response:
{
"q":{
"result":{
"active_start":"1977-01",
"genre":["Rock music","Ska","Reggae"],
"name":"The Police",
"type":"/music/artist"
},
"status":"/mql/status/ok"
},
"status":"200 OK"
}
What's interesting here is that "genre" is a reference to other topics and "active_start" is a primitive datetime type. But how could we possibly know this? And how does MQL know to put the name of the genre topic, instead of the id? The answer is in the "default_property" of a type.
Let's inspect these properties:
{
"q":{
"query":{
"id":"/music/artist",
"properties":[{
"expected_type":{
"default_property":null,
"id":null
},
"id":null,
"id|=":["/music/artist/genre","/music/artist/active_start"]
}],
"type":"/type/type"
}
}
}
And the response:
{
"q":{
"result":{
"id":"/music/artist",
"properties":[{
"expected_type":{
"default_property":"value",
"id":"/type/datetime"
},
"id":"/music/artist/active_start"
},{
"expected_type":{
"default_property":null,
"id":"/music/genre"
},
"id":"/music/artist/genre"
}],
"type":"/type/type"
},
"status":"/mql/status/ok"
},
"status":"200 OK"
}
Note that "active_start" is a datetime, and the default property is "value".. in the case of "/music/genre", there is no "default_property" - when this happens, MQL defaults to the "name" property.
Because MQL has this built in, you get the "right" value if you don't expand the property clause (i.e. if you just put 'null' or '[]')
But MQL has a way of blowing out the default property and automatically asking for the right set of properties - it's {} or [{}] instead of null or []:
{
"q":{
"query":{
"active_start":{},
"genre":[{}],
"name":"The Police",
"type":"/music/artist"
}
}
}
Look at the result:
{
"q":{
"result":{
"active_start":{
"type":"/type/datetime",
"value":"1977-01"
},
"genre":[{
"id":"#9202a8c04000641f8000000000032ba7",
"name":"Rock music",
"type":["/common/topic","/music/genre"]
},{
"id":"#9202a8c04000641f8000000000035edb",
"name":"Ska",
"type":["/common/topic","/music/genre"]
},{
"id":"#9202a8c04000641f8000000000032eca",
"name":"Reggae",
"type":["/common/topic","/music/genre"]
}],
"name":"The Police",
"type":"/music/artist"
},
"status":"/mql/status/ok"
},
"status":"200 OK"
}
For all values, it includes "id" and "type" - for primitive types, it includes "name" or "value" as appropriate.
But what you're doing (expanding all the properties of a particular type) can actually be accomplished with the "*" property, which expands to all properties. I'll let you run the query because the response is quite large:
{
"q":{
"query":{
"*":[{}],
"name":"The Police",
"type":"/music/artist"
}
}
}
Find a topic limited by domain
Let's say you want to find any topic matching your query, and limit the results not by type, but by domain. You can do something like this:
[{
"id": null,
"name": "Malaria",
"type": {
"key": {
"namespace": "/medicine",
"limit": 1
},
"limit": 1
}
}]
Find something limited to Commons types
A related query is to restrict things only to Commons types. This can be done by knowing that the Commons domains are part of the Commons domain category (/category/commons).
{
"type": "/type/type",
"domain": {
"id": null,
"/freebase/domain_profile/category": {
"id": "/category/commons"
}
},
"return": "count"
}
values vs name/id
The shortcut:
"/music/artist/active_start": [{}]
The explanation:
There's a bit of a trick to this. MQL divides the world into "values" and "objects". If you look at the expected_type of a property (try the query
{ "query0":
{ "query":
{ "id": "",
"type": "/type/property",
"expected_type": null
}
}
}
You will find out if a property expects a value (one of /type/id, /type/key, /type/text, /type/rawstring, /type/int, /type/float, /type/boolean and /type/uri) or an object (anything else)
Then you can ask
[{ "name": null, "id": null, "type": [], "optional": true }]
for the objects and
[{ "value": null, "type": null, "optional": true }]
for the values. (the optional: true ensures the query doesn't fail if the property is missing)
[{}] asks MQL to automatically inspect the expected_type for you and insert one or other of the clauses you see above, depending on which is appropriate. It's what we use internally 90% of the time.
Find incoming links
Get all incoming links for a specific topic. Restricted to return only links from objects type topic and user_profile.
[
{
"/type/reflect/any_reverse" : [
{
"guid" : null,
"name" : null,
"type|=" : [
"/common/topic",
"/freebase/user_profile"
]
}
],
"name" : "Repo Man",
"type" : "/common/topic"
}
]
Find any topic with a given date
This query finds topics with a specific date value on any property with an expected type of datetime.
The query demonstrates use of "reflect" and "link" to discover properties with a specific expected type.
[
{
"/type/reflect/any_value" : [
{
"link" : {
"master_property" : {
"name" : null
}
},
"type" : "/type/datetime",
"value" : "1999"
}
],
"name" : null,
"type" : "/common/topic"
}
]
Wikipedia
Find the Wikipedia URL for a topic
Wikipedia URLs are created automatically by substituting the /wikipedia/en_id key of the topic into the following URL:
http://en.wikipedia.org/wiki/index.html?curid=8980330
where 8980330 is the /wikipedia/en_id key.
You can retrieve the /wikipedia/en_id (aka the "wpid") from Freebase using any variant of the following MQL:
[
{
"id" : "/en/barack_obama",
"key" : {
"namespace" : "/wikipedia/en_id",
"value" : null
}
}
]
Fetch full Wikipedia article text
Freebase doesn't store wikipedia articles. Freebase provides the wikipedia key so that you can get the article from wikipedia. For example, /topic/en/radiohead has property "/wikipedia/topic/en_id" : "38252"
that fetches:
http://en.wikipedia.org/wiki/index.html?curid=38252
Wikipedia has several ways to get their full text:
http://en.wikipedia.org/wiki/index.html?curid=38252&action=render
More options are available via their API:
http://en.wikipedia.org/w/api.php
Note that you'll need a way to convert the wiki formatting to valid HTML.
Note also that not all Freebase topics have an equivalent Wikipedia article (Freebase topics should e considered a superset of Wikipedia articles) so make sure your code can handle a case where there is no wikipedia article. This query can be used to see if a Freebase article exists:
{
"/common/topic/alias" : [],
"/common/topic/article" : {
"id" : null
},
"id" : "/topic/en/radiohead",
}
Find distinct co-types
From a question posted to the Developer's email list:
"I'm trying to form an MQL read query that specifies a type (e.g. /business/company), finds all topics of that type, lists all other types linked to those topics, and returns a distinct list of those other types (as per a SQL SELECT DISTINCT... query). Is there an MQL "shortcut" equivalent to SQL's SELECT DISTINCT?"
The trick to this, in MQL, is to turn the query inside out.
You want to find all types that have instances shared with a target type /business/company.
[{
"type": "/type/type",
"name": null,
"id": null,
"instance": {
"type": "/business/company",
"limit": 0
}
}]
(The limit clause is included to improve the readability of the result and to decrease the likelihood of timeout.)
Creating relationships with the connect directive
Since concepts can have multiple types, you will always need to specify the type that you are populating values for. Alternatively, you can use the instance ID and the fully qualified property ID for your query:
{
"query" : {
"id" : movieID,
"story_by" : {
"connect" : "insert",
"id" : personID,
"type" : {
"connect" : "insert",
"id" : "/film/film_contributor"
}
},
"type" : "/film/film"
}
}
and
{
"query" : {
"/film/film/story_by" : {
"connect" : "insert",
"id" : personID,
"type" : {
"connect" : "insert",
"id" : "/film/film_contributor"
}
},
"id" : movieID
}
}
are equivalent queries. The second one is useful when you are populating values for multiple types for an instance.
When you're connecting the film contributor, MQL, unlike the web client, does not infer that if the personID you are adding is of type /film/film_contributor and you should specify it explicitly. If the person is already typed as a /film/film_contributor, MQL will return connect:present.
If you want to programatically find the ECT of a property, use this query:
{
"query" : {
"expected_type" : {
"id" : null,
"name" : null
},
"id" : "/film/film/story_by",
"type" : "/type/property"
}
}
Creating a CVT Compound Value Type
Expanding with the connect directive you can create multiple relationships to form a CVT or Compound Value Type. In the following Example, notice that we are also adding the appropriate types to the dedicator, dedicatee and dedicated work. This is necessary so that everything shows up properly in the Freebase UI.
{
"create": "unless_exists",
"id": null,
"type": "/media_common/dedication",
"dedicated_by": {
"id": "/en/dmitri_shostakovich",
"type": {
"connect": "insert",
"id": "/media_common/dedicator"
}
},
"dedicated_to": {
"id": "/en/saint_petersburg",
"type": {
"connect": "insert",
"id": "/media_common/dedicatee"
}
}},
"work_dedicated": {
"id": "/base/imslp/4931",
"type": {
"connect": "insert",
"id": "/media_common/dedicated_work"
}
}
Flag something for deletion or merger
You can use this query to mark something for deletion:
{
"type": "/freebase/review_flag",
"kind": {"id": "/freebase/flag_kind/delete"},
"item": {"id": null},
"create": "unless_exists"
}
Merges are similar except they have two item IDs and use the kind /freebase/flag_kind/merge.
The schema for this changed in December 2009. The old schema used the following type and property (useful info for querying old votes or upgrading code based on previous schema):
{ // Obsolete
"type" : "/freebase/opinion_collection"
"mark_for_delete" : {"id" : [id of topic to delete here]},
} // Obsolete
Count the number of tasks for Delete or Split
This query will count the number of tasks for deletion.
{
"type": "/freebase/review_flag",
"kind": {
"id": "/freebase/flag_kind/delete"
},
"item": [{
"id": null,
}],
"return": "count"
}
Replacing delete with split or merge for the flag kind will allow you to count other types of flags.
Skip flagged topics
Add this clause to your query to skip topics which have been flagged. It can be made more specific by adding a "kind" parameter.
"!/freebase/review_flag/item": [{
"id": null,
"optional": "forbidden"
}]
Sort by index and timestamp
The order in which a set of objects, such as the properties of type, is returned is essentially random. It is generally based on the order in which items were created, but this is not reliable behavior. By default, the web client sorts objects by index and timestamp. Objects with a null index have not been explicitly ordered.
Use the following query to return all topics associated with a property, sorted by the timestamp of the link created between the topic and property, and including the GUID of the topics.
[
{
"/music/artist/origin" : [
{
"guid" : null,
"index" : null,
"link" : {
"timestamp" : null
},
"name" : null,
"sort" : "index"
}
]
}
]
Traversing through non-reversed properties
Problem:
I was trying to create a find documents that were used as posts in a discussion thread. While there exists the property /community/discussion_thread/post on discussion threads, there is no corresponding reverse property, i.e. there is no property of the form /common/document/XXXXX refers to the same link type. I don't have access to add this reverse property to the /common/document type schema so I couldn't do that, and I needed the document to be at the top (root) of my query, so I couldn'y simply flip the query upside down in order to use the property on the thread.
Solution:
I used the /type/reflect/any_reverse property to reference the link between the document and thread by the master property name as in this example:
{"Q":{
"type":"/common/document",
"id":"#DOCUMENT_ID_GOES_HERE",
"/type/reflect/any_reverse":{
"id":null,
"link":{
"master_property":"/community/discussion_thread/post"
}
}
}
}
Will give me the ID of the discussion thread associated with the named document.
Explanation:
/type/reflect is a "virtual type" on all topics whose purpose is to provide "lower level" access to all of the links on a topic. /type/reflect/any_reverse references all of the incoming (non-literal bearing) links of a topic. The "id":null clause above will be filled out with the id of the thread. "link" refers to the link between the topics, and all links have a "master_property", which refers to the property ID of the link in the forward direction.
Combine queries
I query for ID's belonging to a , after that (extracting IDs with PHP), I query foreach ID what CATEGORY, it is in and whats its name: Now the question is, if it is possible to merge both:
"Query for CATEGORIES (and name of given IDs) of a given "
1. ID an DOMAIN of ,
{
"domid":{
"query":[{
"name":null,
"name~=":"",
"type":[{
"domain":null,
"id":null,
"name":null,
"type":"/type/type"
}]
}]
}
}
2. myIdForKeyword) to ask for CATEGORY of it,
{
"category":{
"query":[{
"/type/type/domain":{
"/type/reflect/any_reverse":[{
"name":null,
"type":"/freebase/domain_category"
}],
"name":null
},
"id":"myIdForKeyword",
"name":null
}]
}
}
Here’s how I built it up.
Query for instances of (literally) “keyword” and their types (with a limit, just for testing):
{
"someids" : {
"query" : [
{
"limit" : 5,
"name" : null,
"name~=" : "keyword",
"type" : [
{
"id" : null
}
]
}
]
}
}
But since we are primarily interested in the types, turn the query inside-out (here the limits aren’t for testing, they are because we only need at least one instance at each level, and help the query run faster):
{
"someids" : {
"query" : [
{
"id" : null,
"instance" : [
{
"limit" : 1,
"name" : null,
"name~=" : "keyword"
}
],
"type" : "/type/type"
}
]
}
}
Now let’s find the domains of those types — but while we’re at it, let’s turn that inside-out, too:
{
"someids" : {
"query" : [
{
"id" : null,
"type" : "/type/domain",
"types" : [
{
"instance" : [
{
"limit" : 1,
"name" : null,
"name~=" : "keyword"
}
],
"limit" : 1
}
]
}
]
}
}
Now, what categories hold those domains?
{
"someids" : {
"query" : [
{
"domains" : [
{
"limit" : 1,
"type" : "/type/domain",
"types" : [
{
"instance" : [
{
"limit" : 1,
"name" : null,
"name~=" : "keyword"
}
],
"limit" : 1
}
]
}
],
"id" : null,
"name" : null,
"type" : "/freebase/domain_category"
}
]
}
}
The answer is “System” and “Arts & Entertainment.”
Perform transitive queries
using the new api, transitive queries are possible
all the lakes in canada
otherwise:
Let's say you want to find all the Freebase users who live in the USA, but they have their locations set to things like "San Francisco" or "New Orleans". How do you ask for "USA, and anything within the USA"?
While MQL doesn't currently support transitive closure, the /location/ location/containedby property model was designed to make the type of query you are attempting to write possible. When creating /location/ location topics the idea is to provide (at least) two levels of hierarchical containment in the containedby property. For example, in the US, cities should specify what county they are in as well as what state they are contained in.
As your query indicates universities should be co-typed as locations and should specify the city they are contained in. Using the two- level of hierarchy containment model, cities should be contained in states, and similarly states should be contained in the US. Therefore, you should be able to get what you are looking for with the query:
Note: the ID for the US is /guid/9202a8c04000641f8000000000959f60
{
"query":[{
"/location/location/containedby":[{
"/location/location/containedby":[{
"/location/location/containedby":[{
"id":"/guid/9202a8c04000641f8000000000959f60"
}],
"name":null
}],
"name":null
}],
"name":null,
"type":"/education/university"
}]
}
Someday, MQL will probably handle transitivity operations - but until then it is possible to construct queries which utilize the containment models based on some minimal assumptions about the data. I am happy to work with you (and others) to make sure the data conforms to these minimal assumptions to meet your needs.
Constrain Numeric and Date Properties
Use >, <, >=, or <= after the property ID.
Example 1:
[{
"type": "/chemistry/chemical_element",
"name": null,
"/chemistry/chemical_element/atomic_mass": {
"mass<": 10,
"mass": null
}
}]
Example 2:
[{
"type": "/film/actor",
"name": null,
"/people/person/date_of_birth>": "1970",
"/people/person/date_of_birth": null
}]
Count results or sort by count
Use "return": "count" or "count": null on the query node you want to count. Note that this can have severe performance effects.
Example 1: Counting the number of outermost results
{
"type": "/language/human_language",
"return": "count"
}
Example 2: Counting the number of inner nodes for each outer result
[{
"type": "/business/company",
"name": null,
"founders": {
"return": "count"
}
}]
Example 3: Sort companies by number of founders in descending order:
[{
"type": "/business/company",
"name": null,
"founders": {
"return": "count"
},
"sort":"-founders.count"
}]
Filter by count
With MQL 1.0, you cannot filter by count. The best you can do is to sort by count and then filter in whatever is consuming the MQL output.
Getting the Language of a String Value
Use [{ "value" : null, "lang" : null }].
Example:
[{
"type": "/location/continent",
"name": [{
"value": null,
"lang": null,
"limit": 5
}]
}]
Match String Properties Against a Pattern
Use ~= after the property ID. Pattern matching operators include ^$*. Wild cards match a single word, but there are implicit multi-word wild cards on the ends of the match string unless the anchor characters (^$) are used. See MQL reference for more info.
Example:
[{
"type": "/film/actor",
"name~=": "Christian *",
"name": null
}]
Matches any actor with the word "Christian" in their name and at least one following word.
Require that Property does not Exist
Use "optional" : "forbidden". MQL also requires that you include some other property in the same query node.
Example: Planets without anything (say, a moon) orbiting them
[{
"type": "/astronomy/planet",
"name": null,
"/astronomy/orbital_relationship/orbited_by": [{
"id": null,
"optional": "forbidden"
}]
}]
Exclude staff/bots from results
Include a clause something like this:
"creator": {
"!/type/usergroup/member": {
"id|=": [
"/freebase/mwstaff",
"/freebase/bots"
],
"optional": "forbidden"
}
}
Note: the above is a simplistic version which won't deal well with anything that uses attribution. To deal with them as well, use something along the lines of the following:
"creator": {
"/type/user/usergroup": {
"id|=": [
"/freebase/mwstaff",
"/freebase/bots"
],
"optional": "forbidden"
},
"key": {
"namespace": {
"key": {
"namespace": {
"/type/user/usergroup": {
"id|=": [
"/freebase/mwstaff",
"/freebase/bots"
]
}
}
}
},
"optional": "forbidden"
}
},
Find untyped topics (or which only have a given type)
This query finds topics which only have the type /common/topic and no other. The t2:type parameter uses two nested "optional":"forbidden" clauses to do the tricky bit.
{
"id": null,
"name": null,
"t1:type": "/common/topic",
"t2:type": [{
"id": null,
"key": {
"namespace": "/common",
"optional": "forbidden",
"value": "topic"
},
"optional": "forbidden"
}]
}
This technique can be extended to find topics which only have a given set of types. For example, because the Award Winner and Award Nominee types don't have any included types, it's common to end up with topics that represent people, companies, etc which aren't typed as such. The query below finds all such topics.
{
"id": null,
"name": null,
"t1:type": {"key":{"namespace":"/award"},"limit":0},
"t2:type": [{
"id": null,
"key": {
"namespace": "/common",
"optional": "forbidden",
"value": "topic"
},
"key": {
"namespace": "/award",
"optional": "forbidden",
"value": "award_winner"
},
"key": {
"namespace": "/award",
"optional": "forbidden",
"value": "award_nominee"
},
"optional": "forbidden"
}]
}
Find if a Freebase user is a part of a user group
This query finds if a freebase user is a member of a given user group
{
id : "/user/jg",
"!/type/usergroup/member": [{
"id": "/en/metaweb_staff"
}]
}
If the query returns null, then the given user is not part of that group (or it is, if the result is not null)
Useful freebase-wide groups are:
- /freebase/badges/topcontributor - users that have earned the 'top contributor' badge
- /freebase/badges/freebaseexpert - users that are considered experts in one or more data domain
- /freebase/odesk - people that are paid to contribute data via ODesk
- /freebase/bots - non-human users that contribute data
- /en/metaweb_staff - people that have been employed by Metaweb at any point in their life
- /en/current_metaweb_staff - people that are currently employed by Metaweb
Search for a value in two different properies (e.g. name & alias)
The following query will return all topics which have the word "kid" in their name or aliases:
[{
"mid": null,
"name": null,
"/type/reflect/any_value": {
"limit": 1,
"value~=": "kid",
"type": "/type/text",
"lang": "/lang/en",
"link": {
"master_property": {
"id|=": [
"/type/object/name",
"/common/topic/alias"
]
}
}
}
}]
