Category Archives: eyeData

eyeData – Coding

What We Worked On

This week, we actually got started coding. Our friends at IQSS set us up with a server and website  http://eyedata.datascience.iq.harvard.ed…). We all set up our personal environments and are working off a git repo. We use Jenkins to integrate our changes nightly.

With our midterm presentation this week, we then decided to split up the project and each of us (Batsheva, Luis, and myself) would develop an area and integrate them for the presentation. Luis is working on the UI; Batsheva is processing the data; I am using d3.js to create visuals.

What Went Well

So far it’s all has been going quite well! We were able to lay out the architecture of the website clearly and divide up the work evenly. We are on track to have a very basic version this week.

What Was Challenging

Setting up the environment for Windows was challenging. I eventually just resorted to setting it up on a VM running Ubuntu.

For me personally, it has be challenging to navigate the various technologies that we are using. Most of what we are using, such as Django and d3.js, are new to me, so a lot of time is spent just learning the language and principles behind them. Luis and Batsheva have previous experience with their modules, but regardless it takes a nontrivial amount of time to work through. Once these fundamentals have been laid out, we’ll have a barebones version of the website that will use some datasets living in the git repo to generate some visuals!

What’s Up Next

Currently, we are all working on our individual parts in order to have something to present at the midterm presentations this Wednesday. Our goal is to have a basic mockup of the UI that we envision, a processed data set, and a visualization created from that dataset. If you’re in the area, feel free to stop by and check us out!

After this week, we’ll probably hook up our website with the Dataverse search API, polish the UI, and automate the creation of the d3.js graphs. Once we have the website’s functionality up and running, we’ll all transition to creating graphs for different types of data.

Until next week,

Alex

Project Update, eyeData

Hi everyone, this is Batsheva from team eyeData!

This week we discussed our project specifications and what data visualization tools will be most useful for us to develop. We decided to focus on differentiating our tools from the already existing ones, for instance having eyeData offer specific visualization tools for each data set based on the metadata that we will get from Dataverse. We think we will start off focusing on survey data, because that is a lot of the information in the Dataverse.

We also talked about the backend of our project. The team at IQSS wrote the backend and put it up on Github, and by our next meeting on Wednesday we will each have it running locally. Once it’s running locally we will start developing the data visualization tools, developing the basic functionality together at our meeting and then each taking on responsibility for certain visualization tools.

Looking forward to this weeks meeting!
Batsheva Moriarty (Harvard College ’16)

eyeData – Planning

Hello there, Alex from eyeData with an update on our progress.

 

Unfortunately due to a miscommunications, our advisers weren’t in town and we were not able to meet with them.

 

However, we have still been hard at work. This week, we have been hard at work building our design doc, which Luis mentioned in our blog post last week. This design doc describes our overall goals and implementation strategies for our final product. The doc encompasses our discussion so far on UI and other aspects of the final product and was a good exercise in scoping out a project and defining milestones, deadlines, technical writing, etc. Furthermore, it was a good exercise in collaborating and dividing up work so that all of us can meaningfully contribute.

 

Without further ado, and since for the most part ya’ll don’t have access to our doc, here is a description of our vision of our basic product. We want to create a standalone website. Users will be able to search for datasets on Dataverse, using the Dataverse API, and then select whichever options they want. They will then be taken to a dashboard where they will be able to select which type of visualization they want. Our most basic goals are handling one dataset and basic graphs, such as bar graphs, scatter plots, time graphs, etc. But our broader vision is to allow users to choose multiple datasets and interact with their variables, as well as create more visually striking and meaningful graphs. Check out the Github page for d3.js and let us know your favorites!

 

Next week, we will actually get our hands dirty coding and developing! We also need to discuss how the work will be split up. Our friend from last week, Raman, will help us set up a server and the necessary tools. We may need to work through some tutorials to familiarize ourselves with the languages we’re using.

 

Until next week!

Alex Wang (College ’17)

eyeData – Getting Started

Hi! My name is Alex and I’m a member of eyeData, a Digital Problem Solving Initiative project working on creating a data visualization tool to integrate with Dataverse, the largest collection of research data led by Mercè Crosas of the Institute for Quantitative Social Science and Vito D’Orazio. The rest of the team is four undergraduates at Harvard College, as well as a more senior member assisting us all the way from the U.K.

Getting started with the project, we gathered to discuss the scope of the project and the direction that we wanted to take the project in. It turns out that there are rudimentary tools existing already, for example Princeton University has a tool called Roper that can generate tables and graphs for the survey results it includes in its database. Similarly, Dataverse already has a feature called TwoRavens that leverages metadata attached to the data sets in order to create graphs with some user interactivity built in. Knowing that some of these features exist, we will take these into account as we meet again next week to discuss our task for the week.

Our task was for each individual to identify some interesting features or graphs that they would like to see implemented in the project. Since we are using d3.js, I’m sure many of us turned to the d3.js github for inspiration, although there are certainly many striking visualizations online to inspire us.

As this is only our first week and we haven’t dived into the programming, nothing has been challenging thus far. But we are all excited to dive into the code and learn! Many of us have prior experience with R, Javascript, and d3.js, the main technologies we will be building with, but this project will require us to expand our knowledge bases of those languages, which I’m sure will be a challenging, though rewarding experience.

Best,

Alex Wang