Tag Archives: altmetrics

The ResearchGate Score: a good example of a bad metric

According to ResearchGate, the academic social networking site, their RG Score is “a new way to measure your scientific reputation”. With such high aims, Peter Kraker, Katy Jordan and Elisabeth Lex take a closer look at the opaque metric. By reverse engineering the score, they find that a significant weight is linked to ‘impact points’ – a similar metric to the widely discredited journal impact factor. Transparency in metrics is the only way scholarly measures can be put into context and the only way biases – which are inherent in all socially created metrics – can be uncovered.

Altmetrics could enable scholarship from developing countries to receive due recognition.

The Web of Science and its corresponding Journal Impact Factor are inadequate for an understanding of the impact of scholarly work from developing regions, argues Juan Pablo Alperin. Alternative metrics offer the opportunity to redirect incentive structures towards problems that contribute to development, or at least to local priorities. But the altmetrics community needs to actively engage with scholars from developing regions to ensure the new metrics do not continue to cater to well-known and well-established networks.


LSE Impact Blog.

Franzen’s The Kraus Project on Altmetrics

Over the weekend I read "The Kraus Project", the newest book by Jonathan Franzen. In it Franzen translates a couple of texts written by  Viennese satirist Karl Kraus roughly 100 years ago in "Die Fackel". The translations are explained with a multitude of explanatory and autobiographical footnotes put together by Franzen, Paul Reitter and Daniel Kehlmann. Kraus in general was sceptic of technological change and Franzen describes him as
"Kraus was the first great instance of a writer fully experiencing how modernity, whose essence is the accelerating rate of change, in itself creates the conditions for personal apocalypse."
And somewhere on page 273 of the book a passage in the footnotes struck me as a noteworthy counterargument to the current altmetrics craze:
The work of yakkers and tweeters and braggers, and of people with the money to pay somebody to churn out hundreds of five-star reviews for them, will flourish in that world. (Kraus's dictate "Sing, bird, or die" could now read "Tweet, bird, or die.") But what happens to the people who want to communicate in depth, individual to individual, in the quiet and permanence of the printed word, and who were shaped by their love of writers who wrote when publication still assured some kind of quality control and literary reputations were more than a matter self-promotional decibel levels?
A video on the book project can be found here. SF

What is impact?

 Metrics are very important. They decide who gets hired and promoted, help us judge the value of research that’s funded, and what new research gets funding. However, it’s important to recognize that impact measures – whether citations, downloads, tweets or shares – shouldn’t be confused with the kind of scientific impact that matters most.

Mendeley, Open Access, and the Altmetrics Revolution: Implications for Japanese Research

Keita Bando, Mendeley Advisor, Digital Repository Librarian Coordinator for Scholarly Communication, My Open Archive on open access, Mendeley and Altmetrics.
“With Japanese researchers, there’s a kind of resistance or barrier to using a creative commons license; particularly, people tend to have the attitude that the rights to a paper are the publisher’s concern, so they don’t think to actively go for a creative commons license themselves,” Mr. Bando states.

Altmetrics Collection

Altmetrics Collection

Altmetrics is the study and use of non-traditional scholarly impact measures that are based on activity in web-based environments. As scholarship increasingly moves online, these metrics track associated interactions and activity to generate fine-grained data, allowing researchers and policy makers to create a higher resolution picture of the reach and impact of academic research.

Check out the PLOS Altmetrics Collection. 

PLOS.


The Promise Of Another Open: Open Impact Tracking

The Promise Of Another Open: Open Impact Tracking

Heather Piwowar:

Over the last few months I’ve been having a great time hacking on an app that reveals open altmetrics stats (and their context) for diverse research products.  total-Impact started in a 24-hour hackathon at the Beyond Impact workshop funded by the Open Society Foundations.  Since then a few of us have been unable to put it down.  I’ll talk about it a bit more in a future blog post [added link, also see here], but you are welcome to read more and play around with the alpha releasenow!

The time is clearly right for this sort of app…several similar ones are emerging now too.

Webometrics and altmetrics: digital world measurements

Webometrics and altmetrics: digital world measurements

Here are some thoughts on altmetrics by Jenny Delasalle.

The problem with web stats and altmetrics is that there are potentially a lot of sources which will all measure the stats for different versions of the same item, or different elements of the same output, in different ways. This sort of thing is a driver for publication in an open access (OA) journal with one canonical copy of an article in just one place online: the so called “gold” route to OA.

It’s a valid argument. One of the main reasons why impact factors are so heavily relied upon is caused by the fact that it is easy to argue against alternatives. The question then is what does a download count say about a paper? Very little I guess. It just indicates how many people have looked at it—it does not say anything about the quality of the content. This assessment can hardly be automated; a crowdsourced solution is probably more feasible. 

Baseball Stats and Impact Factors

Baseball is a game of statistics. Ever detail of every game gets scored, every ball a player hits or misses gets noted and ends up in a database. Baseball fans love getting lost in cascades of numbers, love comparing players and teams. Every baseball fan knows that statistics are an essential element to what makes this game so exciting to follow.

There is an interesting link between baseball and science and that is that science is increasingly becoming a game of statistics, too. And by that I mean that impact factors in many cases have a higher priority than the actual content of the scientific work. In other words: many scientists would prefer a dull paper that hardly anyone is interested in but that is published in a highly ranked journal over a great paper that actually conveys important finding but that does not get published in a top tier journal. That is kind of odd.

Now, there is a noteworthy difference between baseball and science and that is that science is focused on a single measurand and that is the impact factor. In baseball there are dozens of measurands that provide insights into how a player performs in every aspect of the game. Yet, there is not an overall statistic that simply says who is the greatest player. There is no such statistic because it wouldn’t make sense. A great defensive player might not be as great offensively, a great pitcher might not have the stamina of a weaker pitcher … Statistics help fans to understand who is good and who is not, they point to the strengths and the weaknesses of players but they cannot decide who is the most valuable player. Annually, this decision gets made by humans, by the Baseball Writers Association of America, and they never decide unanimously.

In science however, people belief that a single measurand is enough to rank scientists. It is somewhat absurd that the same people whose job it is to create knowledge have created so little knowledge on how to comprehensively evaluate scientific works. Why don’t we have a database that shows how often a journal article gets downloaded, how often it gets mentioned outside the scientific community, how many word a scientist publishes a year, how often he/she helps other scientists, or how many other scientist find an article actually insightful?