Need Help?

Compass API Examples

Last Updated: Feb 20, 2015
 

Compass API examples

 

In this document, we'll run through some quick examples of how to interact with the Compass API programmatically. We'll do so in this nice iPython notebook, so you can copy and run the code if you'd like, but the examples should generalize to other languages fairly straightforwardly.

In [1]:
import requests
import json
In [2]:
EMAIL = 'test@luminoso.com'
PASSWORD = 'test'
BASE_URL = 'https://compass.luminoso.com/api/'
 

Preliminaries over with, we'll need to get authenticated to do anything. Posting username and password to login/ will get us a token we'll submit with the rest of our requests.

In [3]:
login_resp = requests.post(BASE_URL + 'login/',
                           data={'username': EMAIL,
                                 'password': PASSWORD})
 

Cool. Now we just put that stuff in the header and all will be well:

In [4]:
headers = {'authorization': 'Token ' + login_resp.json()['token'],
           'Content-Type': 'application/json'}
 

While we're here, notice that we've also set a content-type header, which tells the server what format we're sending data in. This only matters for requests that send data (PUT and POST requests).

In [5]:
requests.get(BASE_URL + 'users/', headers=headers).json()
Out[5]:
{'count': 1,
 'previous': None,
 'next': None,
 'results': [{'admin': False,
   'name': 'API Test Account',
   'must_reset_password': False,
   'email': 'test@luminoso.com',
   'url': 'https://compass.luminoso.com/api/users/test%40luminoso.com/',
   'permissions': [{'url': 'https://compass.luminoso.com/api/permissions/15/',
     'level': 'readwrite',
     'user': 'test@luminoso.com',
     'account': 'zbs68wtx'}],
   'default_account': 'zbs68wtx'}]}
 

Cool, we can see our user!

One thing to note here: if your user has just been created or had the password reset by an admin, "must_reset_password" will be True, and you will need to change your password (using PUT /users/<email>/password/ or using the website) before most API calls will work.

Let's see what accounts we have access to:

In [6]:
requests.get(BASE_URL + 'accounts/', headers=headers).json()
Out[6]:
{'count': 1,
 'previous': None,
 'next': None,
 'results': [{'url': 'https://compass.luminoso.com/api/accounts/zbs68wtx/',
   'id': 'zbs68wtx',
   'name': 'CompassAPIExamplesAccount'}]}
 

We'll save the account ID so we can use it when creating a project.

In [7]:
account_id = requests.get(BASE_URL + 'accounts/', headers=headers).json()['results'][0]['id']
 

We can get the list of projects:

In [8]:
requests.get(BASE_URL + 'projects/', headers=headers).json()
Out[8]:
{'count': 0, 'previous': None, 'next': None, 'results': []}
 

No projects yet -- let's make one. We're telling the server to expect json, so we'll json-encode the data.

In [9]:
project_data = {'account': account_id,
                'name': 'API example project',
                'description': 'Check out this example!',
                'language': 'en',
                'status': 'active',
                'twitter_cap': 0}
In [10]:
project_info = requests.post(BASE_URL + 'projects/',
                             headers=headers,
                             data=json.dumps(project_data)).json()
project_info
Out[10]:
{'description': 'Check out this example!',
 'name': 'API example project',
 'creator': 'test@luminoso.com',
 'created': '2015-01-20T23:27:28.156981+00:00',
 'account': 'zbs68wtx',
 'status': 'active',
 'twitter_cap': 0,
 'url': 'https://compass.luminoso.com/api/projects/j5v82hkd/',
 'language': 'en',
 'api_url': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/'}
 

Ok, now we've got a project all set up -- we can configure it to follow something on Twitter or load in our own data. First, let's get our URL out and saved somewhere; when we're actually working with the project, all of the URLs will have this as their prefix.

In [11]:
PROJECT_URL = project_info['api_url']
In [12]:
requests.get(PROJECT_URL, headers=headers).json()
Out[12]:
{'usage': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/usage/',
 'listener/keywords': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/listener/keywords/',
 'stats': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/stats/',
 'logs': 'https://compass.luminoso.com/api/logs/',
 'clusters': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/clusters/',
 'config': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/config/',
 'partitions': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/partitions/',
 'topics': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/topics/',
 'messages': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/messages/'}
 

That's our list of available endpoints. (If you got an error, wait a few seconds and try again, the project might take some time to start up.) Without messages we'll have a hard time doing much, so let's go ahead and get some loaded.

In [14]:
putin_messages = json.load(open('putin_messages.json', 'r'))
putin_messages[0]
Out[14]:
{'id': 120209,
  'timestamp': '2015-01-09T09:11:23+00:00',
  'platform': 'twitter',
  'source_id': 'tag:search.twitter.com,2005:553479135348404226',
  'url': 'https://compass.luminoso.com/api/projects/s2g5fvbn/p/messages/120209/',
  'language': 'en',
  'info': {'provider': {'displayName': 'Twitter',
    'objectType': 'service',
    'link': 'http://www.twitter.com'},
   'retweetCount': 0,
   'postedTime': '2015-01-09T09:11:23.000Z',
   'gnip': {'language': {'value': 'en'},
    'matching_rules': [{'value': 'lang:en putin -is:retweet', 'tag': None}]},
   'link': 'http://twitter.com/penguinocI00m/statuses/553479135348404226',
   'body': "i don't check my email and my phone is dead so if you have a 2496 x 3586 picture of putin???",
   'verb': 'post',
   'objectType': 'activity',
   'actor': {'summary': 'Not a bot, no matter how much it looks like it.',
    'twitterTimeZone': None,
    'friendsCount': 9,
    'utcOffset': None,
    'links': [{'rel': 'me', 'href': 'http://420.moe'}],
    'languages': ['en'],
    'verified': False,
    'location': {'displayName': 'twitter.com/penguinod00m',
     'objectType': 'place'},
    'listedCount': 0,
    'id': 'id:twitter.com:2925827213',
    'displayName': '[s4s]-tan?',
    'followersCount': 13,
    'preferredUsername': 'penguinocI00m',
    'favoritesCount': 12,
    'objectType': 'person',
    'postedTime': '2014-12-16T21:09:53.000Z',
    'statusesCount': 756,
    'image': 'https://pbs.twimg.com/profile_images/544962930408579072/wpxQXggx_normal.png',
    'link': 'http://www.twitter.com/penguinocI00m'},
   'twitter_entities': {'symbols': [],
    'hashtags': [],
    'urls': [],
    'trends': [],
    'user_mentions': []},
   'favoritesCount': 0,
   'twitter_filter_level': 'medium',
   'twitter_lang': 'en',
   'object': {'id': 'object:search.twitter.com,2005:553479135348404226',
    'summary': "i don't check my email and my phone is dead so if you have a 2496 x 3586 picture of putin???",
    'link': 'http://twitter.com/penguinocI00m/statuses/553479135348404226',
    'objectType': 'note',
    'postedTime': '2015-01-09T09:11:23.000Z'},
   'generator': {'displayName': 'Not a Bot', 'link': 'http://420.moe/'},
   'id': 'tag:search.twitter.com,2005:553479135348404226'},
  'weight': 10.0,
  'blocked': False,
  'text': "i don't check my email and my phone is dead so if you have a 2496 x 3586 picture of putin???",
  'is_spam': False}
In [15]:
for i in range(0, len(putin_messages), 100):
    requests.post(PROJECT_URL + 'messages/',
                  headers=headers,
                  data=json.dumps(putin_messages[i:i+100]))
In [16]:
requests.get(PROJECT_URL + 'messages/', headers=headers).json()
Out[16]:
{'new_messages': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/messages/?newer_than_id=1905',
 'count': 1859,
 'previous': None,
 'next': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/messages/?page=2',
 'results': [{'id': 1905,
   'blocked': False,
   'weight': 1.0,
   'platform': 'twitter',
   'source_id': 'tag:search.twitter.com,2005:553030130097225729',
   'url': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/messages/1905/',
   'language': 'en',
   'info': {'provider': {'displayName': 'Twitter',
     'objectType': 'service',
     'link': 'http://www.twitter.com'},
    'retweetCount': 0,
    'postedTime': '2015-01-08T03:27:12.000Z',
    'gnip': {'language': {'value': 'en'},
     'matching_rules': [{'value': 'lang:en putin -is:retweet', 'tag': None}]},
    'link': 'http://twitter.com/lovecanada2014/statuses/553030130097225729',
    'twitter_filter_level': 'medium',
    'verb': 'post',
    'objectType': 'activity',
    'inReplyTo': {'link': 'http://twitter.com/RT_com/statuses/553028874205143041'},
    'actor': {'summary': 'Canadian born & raised. Against threats to world peace: Putin, Novorossiya, ISIL. Have seen it before. Trying to see through the bullshit. Are you?',
     'twitterTimeZone': None,
     'friendsCount': 147,
     'postedTime': '2014-09-01T18:09:39.000Z',
     'links': [{'rel': 'me', 'href': None}],
     'utcOffset': None,
     'languages': ['en'],
     'verified': False,
     'displayName': 'Lovemycountry',
     'location': {'displayName': 'Ontario', 'objectType': 'place'},
     'listedCount': 3,
     'id': 'id:twitter.com:2758413141',
     'followersCount': 153,
     'preferredUsername': 'lovecanada2014',
     'favoritesCount': 703,
     'objectType': 'person',
     'statusesCount': 6473,
     'image': 'https://pbs.twimg.com/profile_images/542430145869402114/4-og78oo_normal.jpeg',
     'link': 'http://www.twitter.com/lovecanada2014'},
    'twitter_lang': 'en',
    'favoritesCount': 0,
    'body': "@RT_com Would he drink it if Vladimir Putin was opening the tap? Trust me, it's fine. I had it yesterday (snicker).",
    'twitter_entities': {'hashtags': [],
     'symbols': [],
     'urls': [],
     'trends': [],
     'user_mentions': [{'id': 64643056,
       'id_str': '64643056',
       'name': 'RT',
       'indices': [0, 7],
       'screen_name': 'RT_com'}]},
    'object': {'id': 'object:search.twitter.com,2005:553030130097225729',
     'postedTime': '2015-01-08T03:27:12.000Z',
     'link': 'http://twitter.com/lovecanada2014/statuses/553030130097225729',
     'objectType': 'note',
     'summary': "@RT_com Would he drink it if Vladimir Putin was opening the tap? Trust me, it's fine. I had it yesterday (snicker)."},
    'generator': {'displayName': 'Twitter for iPad',
     'link': 'http://twitter.com/#!/download/ipad'},
    'id': 'tag:search.twitter.com,2005:553030130097225729'},
   'timestamp': '2015-01-08T03:27:12+00:00',
   'text': "@RT_com Would he drink it if Vladimir Putin was opening the tap? Trust me, it's fine. I had it yesterday (snicker).",
   'is_spam': False}, 
   ...]}
 

Cool, messages! We probably have some topics by now, so let's get them:

In [17]:
topics = requests.get(PROJECT_URL+ 'topics/', headers=headers).json()
topics['results'][0]
Out[17]:
{'measurements': {},
 'title': 'Putin condemned the Paris attack, but not before his allies swarmed social media to praise or rationalize it. http://t.co/CzLBYx56UQ',
 'created': '2015-01-20T23:29:16.344020+00:00',
 'blocking': False,
 'status': 'pending',
 'url': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/topics/9/',
 'clusters': [{'url': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/clusters/9/',
   'index': 8,
   'partition': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/partitions/1/',
   'topic': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/topics/9/'}],
 'info': {},
 'time_buckets': {},
 'messages_url': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/topics/9/messages/'}
 

So that's what a topic looks like.

The "measurements" are things like its acceleration, velocity, and message count; the "time_buckets" tell you how many messages have sorted into the topic in the past however many minutes. They aren't included by default because they take some time to look up, but you can get them by specifying stats=true:

In [18]:
requests.get(topics['results'][0]['url'],
             params={'stats': True},
             headers=headers).json()
Out[18]:
{'measurements': {'velocity': 0.01738519500208872,
  'count': 48,
  'acceleration': -0.0063431926868929706},
 'title': 'Putin condemned the Paris attack, but not before his allies swarmed social media to praise or rationalize it. http://t.co/CzLBYx56UQ',
 'created': '2015-01-20T23:29:16.344020+00:00',
 'blocking': False,
 'status': 'pending',
 'url': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/topics/9/',
 'clusters': [{'url': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/clusters/9/',
   'index': 8,
   'partition': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/partitions/1/',
   'topic': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/topics/9/'}],
 'info': {},
 'time_buckets': {'end_time': '2015-01-20T23:25:00+00:00',
  'interval': 300,
  'start_time': '2015-01-20T13:25:00+00:00',
  'counts': [0,0,0,...]},
 'messages_url': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/topics/9/messages/'}
 

Let's see some messages from this topic:

In [19]:
messages_url = topics['results'][0]['messages_url']
requests.get(messages_url, headers=headers).json()
Out[19]:
{'count': 17,
 'previous': None,
 'next': None,
 'results': [{'id': 1875,
   'blocked': False,
   'weight': 1.54883611482446,
   'platform': 'twitter',
   'source_id': 'tag:search.twitter.com,2005:553036133714907136',
   'url': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/messages/1875/',
   'language': 'en',
   'info': {'provider': {'displayName': 'Twitter',
     'objectType': 'service',
     'link': 'http://www.twitter.com'},
    'object': {'id': 'object:search.twitter.com,2005:553036133714907136',
     'postedTime': '2015-01-08T03:51:03.000Z',
     'link': 'http://twitter.com/LowAnimalSpirit/statuses/553036133714907136',
     'objectType': 'note',
     'summary': 'Elliott paris northeast putin'},
    'postedTime': '2015-01-08T03:51:03.000Z',
    'gnip': {'language': {'value': 'en'},
     'matching_rules': [{'value': 'lang:en putin sample:50 -is:retweet',
       'tag': None}]},
    'link': 'http://twitter.com/LowAnimalSpirit/statuses/553036133714907136',
    'twitter_filter_level': 'medium',
    'verb': 'post',
    'objectType': 'activity',
    'actor': {'summary': 'live model of HFT dealing in words sourced from global news for virtual profit whilst speculating on their usage - tweeting speculative headlines',
     'twitterTimeZone': 'Amsterdam',
     'friendsCount': 34,
     'postedTime': '2014-07-09T17:04:16.000Z',
     'links': [{'rel': 'me', 'href': 'http://www.bannerrepeater.org'}],
     'utcOffset': '3600',
     'languages': ['en'],
     'verified': False,
     'displayName': 'Low Animal Spirits',
     'location': {'displayName': 'Banner Repeater', 'objectType': 'place'},
     'listedCount': 6,
     'id': 'id:twitter.com:2613884546',
     'followersCount': 137,
     'preferredUsername': 'LowAnimalSpirit',
     'favoritesCount': 9,
     'objectType': 'person',
     'statusesCount': 23264,
     'image': 'https://pbs.twimg.com/profile_images/512722414833111041/QOEuEhvI_normal.jpeg',
     'link': 'http://www.twitter.com/LowAnimalSpirit'},
    'twitter_lang': 'en',
    'favoritesCount': 0,
    'body': 'Elliott paris northeast putin',
    'twitter_entities': {'hashtags': [],
     'symbols': [],
     'urls': [],
     'trends': [],
     'user_mentions': []},
    'retweetCount': 0,
    'generator': {'displayName': 'LowAnimalSpirits',
     'link': 'http://artgeometry.com'},
    'id': 'tag:search.twitter.com,2005:553036133714907136'},
   'timestamp': '2015-01-08T03:51:03+00:00',
   'text': 'Elliott paris northeast putin',
   'is_spam': False},
   ...]}
 

Say we like this topic and we want to save it. That's easy to do as well:

In [20]:
topic_url = topics['results'][0]['url']
requests.put(topic_url,
             headers=headers,
             data=json.dumps({'status': 'active'})).json()
Out[20]:
{'measurements': {},
 'title': 'Putin condemned the Paris attack, but not before his allies swarmed social media to praise or rationalize it. http://t.co/CzLBYx56UQ',
 'created': '2015-01-20T23:29:16.344020+00:00',
 'blocking': False,
 'status': 'active',
 'url': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/topics/9/',
 'clusters': [{'url': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/clusters/9/',
   'index': 8,
   'partition': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/partitions/1/',
   'topic': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/topics/9/'}],
 'info': {},
 'time_buckets': {},
 'messages_url': 'https://compass.luminoso.com/api/projects/j5v82hkd/p/topics/9/messages/'}
 

Lastly, while there's much more you can do with this API (see the API documentation), it's worth just verifying that you can get message statistics for the project as a whole as well, so you can compare a topic to the overall volume we received:

In [21]:
requests.get(PROJECT_URL + 'stats/', headers=headers).json()
Out[21]:
{'measurements': {'velocity': 0.35731219982419127,
  'count': 1950,
  'acceleration': -0.2138672581722336},
 'time_buckets': {'end_time': '2015-01-20T23:30:00+00:00',
  'interval': 300,
  'start_time': '2015-01-20T13:30:00+00:00',
  'counts': [0,0,0,...]}}
 

Down here at the bottom, let's delete the project we've created so we can make sure this all runs smoothly next time:

In [22]:
requests.delete(project_info['url'], headers=headers)
Out[22]:
<Response [204]>

More Support

6560c840000970c2e652c21ccafb3167@luminoso.desk-mail.com
http://assets3.desk.com/
false
desk
Loading
seconds ago
a minute ago
minutes ago
an hour ago
hours ago
a day ago
days ago
about
false
Invalid characters found
/customer/en/portal/articles/autocomplete