Archive for September, 2009

Science Hour with Leo Laporte & Dr. Kiki

September 29, 2009

Just recently Dr. Jason Hoyt of Mendeley and Pete Binfield of PLoS ONE where invited to the podcast Science Hour with Leo Laporte & Dr. Kiki. During this one hour podcast they discussed scientific publishing on the internet. Some interesting topics were discussed, I will summarize them in this blogpost.

Why is publishing important in science.

Currently their is an attitude of “publish or perish”, which means their is pressure to publish to get or maintain a career in academia. A consequence is that people want to publish first and sometimes this causes fraud or a publication with false data. To avoid this their is the concept of scientific publishing, it contains the following phases:

  • records premise of a discovery (first to publish = guy who thought it up)
  • certification (peer-review, analysis of correctness),
  • dissemination,
  • archiving

One would think that this would be easy using the internet as a tool, we don’t need a publishing company any more to publish. But a problem arises how do we give credibility to a publication, is peer-review a correct way to do this?

Commenting

PLoS ONE tried adding a commenting system, this had a surprising result. Commenting on article was possible by annotation on text or sentence, star rating on article, …

The problem was that is wasn’t very well used, only a few comment. The discussion took place in a broader community like on blogs, chat, …

Traditional publishing

Another problem is that traditionally a researcher would write an iteration paper, receive comment on it and write a new one. If we look at the current technology this seems ridiculous. The result or conclusion of your paper isn’t as important any more, the first worry should be to get your data out in the world. Often publication don’t even contain the full data.

Another tradition in publishing is in peer-review, the current technology seems to require a quicker way to put your paper online and review it. What happens now at PLoS ONE is that they check if it contains good science. Other aspects like impact factor is decided by community.

At the same time I think this is a problem because some scientist are critical about online communities and might not think their is a high value to the online determined impact factor. This is why online tools should prove that the systems they use are just as good in determining values like impact factor as the old system.

How to get the respected scientists online.

At the moment younger scientists are forced to publish in traditional way because of pressure to get a grant, tenure track, … An other reason is social reluctance the way career structure of academics is setup.

At the moment I think this seems to be a social problem, which might solve over time. But might be helped by putting some journals online only. Of course again the issue of online value of a paper should be addressed.

Too much information

A lot of people are publishing, will we be able to find the right information? Traditionally journals would filter the information flow, disadvantage was that some good articles weren’t discovered. Filtering information online is done by an algorithm, Mendeley has another approach. Mendeley allows articles to be discovered at an individual level, lower impact articles can still be found.

When data is published online the find ability and speed of searching is a lot higher than in the traditional way. Another way to find your way in the large number of publications is by tagging.

Impact factor

Another problem with online publication is the impact factor, users with a high impact factor are those who already have established an impact factor. This forces younger people to go to places where they can get an high impact factor because of the academic pressure to publish. They end up in the old model where they want to get published in an journal with an high impact factor.

Mendeley wants to get rid of this by introducing impact factor at the article level, when doing this the article itself gets examined. To do this Mendeley got some good algorithms.

Article level metrics

An interesting point made in this podcast is about article level metrics. PLoS ONE is pioneering in article level metrics. This is an indicator on an article containing: number of citations, number of social bookmarks, number of blogposts about the article, … Recently they introduced usage data this is information like number of downloads, page views, … Their seems to be a reluctance to make this level of data open. You can refer to these metrics as social media metrics.

To me this is very useful data as it gives you an overview of impact and allows other analysis on the data. But in the podcast they mention we should also focus on standardization so that in the end we have the same metrics everywhere.

Conclusion

I think this podcast contained some relevant discussion it made me familiar with the “publish or perish” tradition. A lot of problems to get people to use online publication seems to involve getting people out of this tradition. The solution isn’t clear at the moment but as people in the podcast suggest, the focus should be more on the article level. To do this we need to find a way to give a value to article based on their impact factor, citations, … As my thesis will involve an open repository I think it could be a nice feature to at some kind of impact factor or article level metrics to the papers. This would offer users a good way to discover new content. It also offers them a way to decide what they think is of great value for science for example by a ranking system, commenting, …

Michael Nielsen about The Future of Science

September 29, 2009

In this following blogpost I’ll try to summarize a blogpost I read about “The future of Science”. This blogpost contains a lot of information, I’m going to focus on the parts that would be of use when I’ll have to implement social features myself.

In his blogplost Michael Nielsen starts with describing the history of science and research. The system of publication hasn’t changed much the last 300 years. But the internet offers us a great way to handle science more openly and a sort of online memory. He predicts that the way we do science will change more in the next 20 years than in the past 300 years.

Part I

In the first part he explains some of the benefits of internet for science, problems, … .

How can the internet benefit science?

It could improve the way we do science by either

  • Online tools as a way of expanding the range of scientific knowledge that can be shared with the world. Example: physics preprint arXiv who allow researchers to share preprints of their papers without a long delay and more recently ResearchGate that allows to share papers based on self-archiving agreements.
  • A change to the process and scale of creative collaboration itself, a change enabled by social software such as wikis, online forums, and their descendants.

Failure of science online

He also show some examples where the idea of online science has failed, an example closest to my thesis is the failure of online comment sites. The example he gives is: Nature’s 2006 trial of open commentary on papers undergoing peer review at Nature. This was not a success because most of the people wanted to read reviews of the papers but not comment on the papers.

One of the reasons he points out is that there are few incentives for people to write such comments.

Achieving extreme openness in science

In the examples mentioned by Michael Nielsen failures are all caused by a surprising reluctance to share knowledge that could be useful to others. He tells us we should aim to open scientific culture where as much information as possible is moved out of people’s heads and labs, onto the network, and into tools which can help us structure and filter the information. This means everything – data, scientific opinions, questions, ideas, folk knowledge, workflows, and everything else – the works. Information not on the network can’t do any good. To achieve a kind of extreme openness.

By this he means: making many more types of content available than just scientific papers, not just the end result but also the creative process.

I think this is a good goal but not realistic because like in all other fields, a creative process is something individual and people are even less eager to share this kind of information. But in my opinion we should offer the tools for this creative process which will offer a way to work online and once they do the creative process online they might be willing to share it.

How can we open up scientific culture?

Michael Nielsen tells us that to create an open scientific culture that embraces new online tools, two challenging tasks must be achieved:

  1. build superb online tools;
  2. and cause the cultural changes necessary for those tools to be accepted.

A mistake that is often made is to focus on building tools, with cultural change an afterthought. He tells us to develop such tools requires a rare combination of strong design and technical skills, and a deep understanding of how science works.

The second task seems to be the most difficult, Michael Nielsen suggest for the people building the new online tools to also develop and boldly evangelize ways of measuring the contributions made with the tools. This is an interesting way of looking at building tools because if using a tool is not recognized in the scientific community why even bother spending time using it.

A success story is the arXiv and SPIRES story in which not long after the arXiv began, a citation tracking service called SPIRES decided they would extend their service to include both arXiv papers and conventional journal articles. Because of the quality of the service it is now used by hiring committees to evaluate candidates.

Looking at this success story and the two tasks of Michael Nielsen I think a good online tool start with good quality. But it doesn’t stop there it should be able the measure the contributions made by the tool!

Part II

In the second part he starts of with the problem of collaboration. Often people have we a small group of trusted collaborators with whom we exchange questions and ideas when we are stuck. Unfortunately, most of the time even those collaborators aren’t that much help. They may point us in the right direction, but rarely do they have exactly the expertise we need.

The question he asks is whether it is possible to scale up this conversational model, and build an online collaboration market to exchange questions and ideas, a sort of collective working memory for the scientific community? An extremely demanding creative culture already exists which shows that such a collaboration market is feasible – the culture of free and open source software.

Next the author describes two embryonic examples which suggest that collaboration markets for science may be valuable. One of the examples is FriendFeed, it can be used for collaboration. But a problem like a lot of online tools is again, the lack of widely accepted metrics to measure contribution.

To end Michael Nielsen gives some examples of inefficiences in the current system of collaboration. Like scientists having problems in a project that is outside their field but could easily be solved by a colleague at the other side of the world. This problem can be solved by using the internet, but another problem comes to mind. For scientist to work there has to be a great deal of mutual trust. An ideal collaboration would have elements like metrics of contributions, impact of work, archiving of contributions, …

Opinion

It is clear that the author is trying to say that their is a lack of metrics of contributions and impact of work in the current tools available. I agree with the author on this point because looking at some of the social networks earlier this information is present. Having more information about how many times a paper is cited or a is mentioned in a science blog could give a great indication of the impact of a work. Creating a value for online work in science would in my opinion be a great incentive for scientist to share content online. One could see this as a challenge for the future.

It is a strange fact that often social networks like Facebook and Linked in are used to check if someone is capable for a job. But when it comes to science their is no equivalent, again their is no value to the online work of a scientist.

Another thing he points out is the difficulty of getting the scientist out of their old culture of publishing, for this reason tools that are developed should have an even higher quality and usability to convince people to use it.

Mendeley the Last.fm for academia

September 24, 2009

Mendeley was created with a minimalized social aspect in mind, this is because often scientist are afraid of social networks. By offering a good organizing tool scientist might be more eager to start using the software and the social features that come with it.

Mendeley is based upon the Last.fm model, it exists of a desktop and a web application. Simply put the desktop application allows you to organize your research files like you would organize your music files. You can create collections, rename, search, tag, …

The application has a science scrobbler which extracts metadata from your files and add it to Mendeley’s database. This speeds up the process of analyzing data like citation. To make the life of a researchers easier Mendeley also offers plug-ins for Word and OpenOffice which allow the user to automatically generate reference lists.

Mendeley is not all about social networking but it’s got some great social features. The desktop application recommends related papers to the ones in your library. And if you choose to share your research, Mendeley offers some nice social features. You can share what you are reading and recommendations. Next to these features Mendeley offers collaboration. Once you share a collection of papers  you can collaborate: you can edit, annotate(commenting) and insert citations to a document.

Next to the desktop application which allows you to synchronize your library at every computer, there is a web application. The online access offers you to create your own profile but most importantly it allows you to manage your library, view statistics about trends in Mendeley’s database, your library and your  publications. The statistics are visualized mostly like they are on a Last.fm profile, trends are easy to spot. The online account makes it possible for the user to sync citations from CiteULike. There is a webimporter to import documents from a large amount of websites, this is of course a great research tool.

Mendeley seems to be a great success, I think this success is mainly because of the scrobbling model from Last.fm. Offering scientists a great and easy way to organize their research will make them more eager to use the software. This also means they might be triggered to use some of the social features offered. It also seems to be a fast way to add papers to a repository, when we look at open repositories people are hesitant to upload their papers because of the complexity or high effort rate to do this. The desktop application makes this process easier but it helps the researcher in his activities. Mendeley tries to addresses the problem of getting researchers to share but doesn’t offer any groundbreaking social features.

Overview of social features in Science 2.0 platforms

September 14, 2009

It’s been a while since I my first blog post but I’m still alive and I’d like to tell you about some of things I’ve done lately. During the last week I’ve started with the first part of my literature study. I’ve been looking at some of the Science 2.0 platforms which try to introduce online social networks for sharing papers, ideas, … . I’d like to give you an overview of the social features they offer in this blog post. I’ve done this to let me get a better overview of social features that might be useful to add to repository software.

In my next blog posts I will be tackling some other aspects of Science 2.0 platforms like desktop applications, online citation managers, … I’d like to review all of these to get some knowledge of the social features they might offer or could inspire me to create.

To start of here’s a list of the sites I’ve been looking at:

Profiles and groups

Almost all of the web based Science 2.0 platforms out there offer the user a way to create his own profile, nothing new there. The researcher is allowed to create his own profile with information like the institution he’s working at, research field, … This offers people in the network to get to know who they are interacting with, but also offers an opportunity for to link people with same interests from over the whole world.

Some of the sites offer a process to complete your profile by importing contacts, … and some even automatically recognize friends and propose them this is done by Epernicus. Important with this last feature is that your profile is as complete as possible. This all sounds great but still someone can act on behalf of someone else, some of the platforms try preventing this by only allowing people with mail addresses from schools, universities, …

An other basic social feature is joining, creating, … of groups this allows for interaction. Groups can be made public or private.

Recommendations

When uploading or looking at a document almost all sites offer recommendations for the user. Another part of recommendations is being able to recommend documents to others and to save those recommendations.

Collaborations

In offering a user ways to share documents working together on a document seems to be a natural next step. But this is a service that is harder to implement than the previous social features. Next to being more difficult there are also more ways to do this.

Documents and groups

A more classic approach taken by some sites is allowing a user to upload documents to his own map and share these with others. Others can in their turn edit, upload and share the document again. This is most of the times related with a group of users, this approach doesn’t seem to be the easiest.

Wiki’s

Most people are already familiar with wiki’s and it’s often used by groups of researchers in labs. So some of the sites just implement wiki’s into their site as a collaboration services. The benefits are that is a very well known principle and easy to implement.

Google Doc system

Some sites even go as far as implementing their own Google Doc like system, as for development this seems a very intensive process. On the other side the system of Google Doc’s is a very good one, allowing people to work on the same document.

When it comes to collaboration an approach like a Wiki seems the best offering an easy way to put a lot of information in a structured way online. Of course with the arrival of Google Wave this might all change.

Visualization

Visualizations in the sites doesn’t seem to get a lot of attention, possibly because they don’t see it as an immediately productive tool for the researches. At first hand this might be so because it’s not a thing we associate with doing research, but it might offer some interesting results. This is shown by Academia.edu and Sci Link, the last one offers a tree of science in which visually maps relationships between people.

Your own dashboards

Personalization is one of the social features offered by some of the Science 2.0 and this is likely best done by a personal dashboard. It allows the user a easy way to access specific services of the site and community. The development of own widgets is sometimes even allowed.

Blogging

Another way of offering interaction is of course offering a blog for every user, this is mainstream in all the services I’ve seen like profiles and groups.

Comments

A user is able to comment on blogs, documents, events, … this allows for interaction on all fields.

News feeds

Offering a user a quick view of what is going on in the community or research field can be done by offering a news feed. This feed can be generated by the service that looks at the profile of the user and gets information the user might be interested in. The choice of what a user might be interested in needs to be right, because a user doesn’t want to see a feed with useless information. Offering a user to integrate news feeds from other websites is also an option that is implemented by some sites.

Academia.edu seems to have focused on news feeds in it’s development, it offer feeds about events, published papers, … Building up a good profile is important in this case. But this approach gives a very personal feeling to the service. Ologeez more or less does the same thing and collects information from groups, wiki’s, … a user is interested in and shows them in a MyWorld page, which gives the user an overview of recent activities.

Another interesting feature of Academia.edu is the Twitter-like following system which allows a user to follow someone and to receive information about them in the news feeds. Yet another Twitter-like feature is implemented by SciLink, SciLink allows for you to share with your friends which paper you are reading at the moment.

Events

As mentioned above, users are interested in events. Some sites take a Last.FM like approach allowing users to look up events, join and review events. Next to events ResearchGATE allows for users to plan meetings.

Managing citations

Some sites offer users to manage their citations, users are able to share them with others, to access them online, … 2Collab takes a different approach than the previously mentioned systems, it is build around bookmarks like Delicious. It let’s people share, manage, discuss, … bookmarks they make.

Conclusion

Most of these social networks try to do more or less the same thing, putting some of their focuses differently, some might concentrate on collaboration and others on searching and sharing of documents. All offer a social network around this all, but there still seems to be little to no integration with existing services like Twitter, Facebook, … which all have a larger audience.

The social features I’d like to see added to repository software should make adding and editing papers, journals, … easier. So managing your citations is a helpful feature for anyone writing his paper.

An easy way to collaborate online would certainly increase activity from users of the repository software. It offers researchers to work together over a greater distance and a place to meet. It’s also easier to have a place to collaborate and publish at the same time. Visualizations offer some great insight to researchers about their papers, people they work with, … but the question I’m likely to ask is, will people frequently use this feature?

News feeds are offered by some of the social networks are interesting and recent developments in this area have been made. Like the lifestreaming service offered by FriendFeed which allows users to share content from different services like Flickr, Delicious, … would likely be a good social feature to share information between users. It gives the user a quick view of what is going on in the community, an easy way to share, …

Other social features like dashboard, event, comments, … are common to most social networks scientific or not. People who’d like to use a social network will most like expect them to be available. Feedback on this post (things I haven’t mentioned or skipped over to quickly) are welcome in the comments.

In my next blog post I will talk about some Science 2.0 platforms that use applications on the desktop instead of a website.