Notes of Interview
Project: IDRC Data Sharing Pilot - https://doi.org/10.3897/rio.2.e8880
Date: 24 September 2015
Location: Paris, France - Research Data Alliance Meeting
Interviewer: Cameron Neylon
Interviewee: Carly Strasser
Interviewee Affiliation: Speaking in a personal capacity with experience as researcher

Audio: 
../Interview_Audio/Interview-24_09_2015-13_57_40-Carly_Strasser.wav
../Interview_Audio/Interview-24_09_2015-13_57_40-Carly_Strasser.flac

With respect to the NSF data sharing policy...

My experience with data sharing policies is primarily with the NSF. In 2011 they enacted the data management plan requirement and that was really the first stab they had taken at trying to promote access to data and research products more broadly. So that particularly policy is still iterating and in my experience it is rather weak, only because it is generally not really followed up on and so I was actually on a couple of panels for proposal review at the NSF while the requirement was in place and no-one really read or looked at the data sharing or data management plans that were sent in along with the proposals even tho they were required.

2:37 

It was more of a box that people thought of as being checked so it turns out to be really challenging to get grass roots organisation like the NSF who really relies on existing scholars to evaluate work to get them to evaluate that work if they don't actually know anything about data sharing and how to do that well. So it's been really interesting as a person who is both prepared DMPs for myself and for people who are affiliated with the university of california where i used to work but also to actually see it from the inside, from the NSF side that in fact its really challenging to get people to care about those things at that level.

3:22 

So the weaknesses with that specific data policy. Is it compliance, tracking, where is

 I think its all of the above. I think the NSF honestly thinks of it as a first step and they recognise that they don't have compliance measures in pace. They don't have a way of making sure this is getting done. They have the rather empty threat that if you turn in a proposal in a couple of years, that they could in theory look back and see if you did the things you said you were going to do in a previous grant DMP. And so that threat is there and it something they could actually do, its something reviewers could do. But the reviewers aren't getting guidance on evaluating plans, they're not getting guidance on evaluating plans on previous proposals that somebody has submitted. So in that sense there is no compliance checking, there is no tracking, and more broadly the US is not  very good at providing infrastructure for these sorts of things.

4:34 

So when a researcher has to write the DMP they don't know that  there are resources for them, they don't know that they can maybe go talk to a local librarian about it and on top of that they don't know what any of the words mean...like "metadata". And those things that the data sharing plan asks you to provide, they don't know how to provide those well. Its a huge problem because maybe they say something in their plan, and they don't know what it means, or they know what it means but its the wrong thing. So they say they're going to archive their data by putting it on a hard drive and they'll leave that with the university when they finish graduate school. Well that's not a data sharing plan but its something you can write down and put in a data sharing plans and if no-one is checking then there is nothing stopping you from making that your total and complete plan. So I think that's the major weakness at least from the perspective of NSF.

5:32 

Are there some things within that plan/program that have been surprising that have been strongly useful in driving adoption.

So the question is has something surfaced that was actually useful?

The best thing about the DMPolicy for the NSF is that there is a policy and that researchers now have to pay attention. Even if they don't write a good one. Even if they don't have any idea what they're doing they at least know that it exists. And that they should write it. And that in theory thats an important component of a proposal.

[...]

6:44 

So the best thing about the plan is that researchers know that they should think about data before the end of the project. Thats the biggest component that I think is valuable about the NSF policy. And certainly its brought attention to institutions in the US as well.  They actually are very aware of the requirements that the NSF has for making products available and they're anxious to make sure that they comply and that the people writing grants from their institution comply as well. So institutions knowing that this is an issue and researchers knowing that this is an issue. Those are all huge benefits.

7:22 

Looking at other policies, are there examples you would point out that are particularly good.

I really like what the Gates foundation has enacted. Their grantees are requires to have very solid plans in place. They're required to have all of their information open and available. There are specific time requirements on when that needs to happen by. There are specific guidelines about what it means to make a publication open or what it means to make data open. And that's huge. Thats exactly what researchers need. They need a formula to follow. Because they don't know about these things, and if you give them vague guidelines they're going to do vague things. And so in the case of the Gates they've done a really good job of laying out what they want and providing researchers with a bit of a roadmap on how to get there and of course they're a funder so they can low those rules down and people are actually going to pay attention to them.

8:46 

So I'm really excited about what they've done and I think its really progressive and I would love other funders to do something similar.

8:55 

So the reverse questions. What are particularly bad elements of policy. How do you navigate the balance between letting communities sort themselves out or dictating.

I think for me, the idea of letting communities sort themselves out is for lack of a better word is kind of naive. I just don't think researchers know about this and if we expect them to know about it, we're not going to get anywhere very quickly. We're not going to get it right. So we're not going to get it right first time. There will need to be some iteration, and maybe it will start off as a box ticking exercise but now they know what the difference is between an open access publication and non open access publication. Now they know that they're supposed to share their data somehow and maybe somebody is going to look at it and notice that its really poorly managed and so now next time they do data, they're maybe going to do it a little bit better. 

So I think if we have those box ticking exercises there is value in that. I don't think that its fair to say that nothing good will come of that because at least we're stepping in that direction and at least if people start to have opinions about how to share data that would be fantastic. If we had arguments amongst research communities about what repositories is best to share your data I would love to have that problem. But now we don't because nobody even knows, they can't even name one repository, never mind have a debate about which one is the best for that type of data. So yeah, I think it is hard to strike that balance and maybe we don't need to worry about that just yet.

10:31 

Whats not working really well? Other things that strike you as not helpful or...

Right so I think in the case of the US, the NSF general policy of avoiding the funding of infrastructure is damaging, and I think that in general more funders should fund infrastructure projects. Especially given how rapidly the research world has changed in the  last 15-20 years we're working with a completely different system now and if we don't support the infrastructure we are going to have these very nuanced things that are popping up for different communities and for different groups. They're not going to be interoperable and we're just going to magnifiy the problems we have in terms of interoperable data, and in being able to work together efficiently. I think that's one of the biggest barriers, is the lack of funding for basic infrastructure.  And infrastructure is such a loaded word mostly because its a boring word so the challenge is how to describe infrastructure in a way so that people understand its importance and its value and how its potentially a really exciting thing to actually engage in funding because you can make really rapid progress with a good infrastructure grant that maybe five other grants that aren't infrastructure couldn't even come close to having that same level of impact. So that's a huge barrier I think.

12:13 

What are the outstanding challenges/next big problem in policy design

In terms of creating policy, the things we really need are buy in from higher up and so I think we have all the basic components more or less floating about, you just need to figure out how to get them all together and make them cohesive but also to describe what we want from researchers in a clear way and keep reminding ourselves that they don't live in this world of data policy so we have to make it really easy for them if we want them to comply with something we have to make it really easy for them because it doesn't align with the kind of incentive structure, so that's the biggest challenge for policy is figuring out how to align what we want the researchers to do with the reasons why they're going to do it and the reasons we're going to get them motivated to do it and thats particularly challenging if they don't know how to do any of the things that we need to get done to share their data and make it open.