Research is tricky. If we knew in advance that something was going to work, it’s probably debatable whether it is ‘research’ in a sense that really matters for your thesis. I mention this because, for most students, they are going to create a lot of different data files in the process of finding out what ‘works’ for their project. This post is about why you want to spend some time thinking about, and managing, your spatial data as you travel through your candidacy.
If you are already on the journey, have a look at your spatial data. Go on, we’ll wait for you…
OK, be honest, did you see any files with a name like:
final_Final_reallyFinal_ThisIsIt_iMeanIt_2.shp? (I swear I didn’t make this one up!)
What about “output22” or some variant of “this_one”?
If you did, all is not lost. It is, however, probably a good time to sit down and have a chat with yourself about what you are doing to your research data.
If you are just starting out on your research journey, or need to pause and take stock of where your spatial data is at, hopefully, the following tips will help you.
Tip One: Start with a suitable structure
Get some buckets in place. It’s never too late, and an excellent place to start is with these three folders at the very top:
AsSupplied: This is a place to put copies of all the data that you got from other sources – in the exact format you got it from the other source. And then you don’t touch it!
This way, if something ‘interesting’ happens to the working copy, you don’t have to annoy the people who made it available in the first place for another copy. The even better bit is that, if the ‘interesting’ was more of a <smack forehead> moment – you are the only one who needs to know you had to back up a little to get going again!
Workspace: The keyword in this folder is ‘work’. This is where you perform all your researchy goodness. Test out those ideas that you have and see if they work. This is where things get messy though – so there will be more tips for managing this part in future posts 🙂
ResearchOutputs: You know that moment when you realise you just completed something quite important and a little bit clever? That’s the time when you might want to make some notes about what you just did and store it somewhere particular. It may not be a final layer, but if it is an importantone – put it in here!
I know it feels like you will remember forever – it was seriously that much of moment, but experience suggests that those crystal clear understandings of how you just made that awesomely smart thing happen will fade. Take a copy of the fantastic new dataset and put it here. But don’t stop there! Visit ‘Creating Your Own Metadata’ and make some metadata for your new creation & put a copy with the new dataset – you won’t regret it. And if you come and ask me in six months time how you made it because you didn’t make metadata and can’t quite remember, all I can do is shrug at you and look sad. So let’s both avoid that!
Tip 2: Do regular housekeeping
Workspace folders are wily beasts. You need to keep an eye on them. They get chaotic quickly, especially when you are in the throes of research inspiration. Have a think about how often you should sit down and actually look at what’s happening in this folder.
Is there something lurking here that should be copied over to the ResearchOutputs folder and given some metadata?
Has some original source data not been copied into the AsSupplied folder because you were so keen to get started it just slipped your notice?
An appropriate recurrence interval will depend on your work style and project size. It can also be influenced by where you are at in your candidacy. Make time to work on your project as well as in it, and your final stages before submission will feel a little more sciencey and be less reliant on fuzzy memories of stuff you did ages ago…
Tip 3: Ask for help if you need it
If the situation has gotten away from you, ask for help. You will have to do the tidying up – it is your mess after all – but we can help you with some advice on how to tackle it and work out how often you need to be doing your Tip 2 Housekeeping.
Spatial data file management gets more comfortable with practice. In future posts, I’ll write up some tips for managing file names while you are in the process of testing a bunch of ideas. Unsurprisingly, having a plan is key to making this more straightforward and there are techniques that can help you manage your way through.
In the meantime, if you didn’t go and check your filenames back when you said you did at the beginning of this post, make some time to do this soon. It is a straightforward way to check the temperature of your spatial data management situation and catch things before they get too hot to comfortably handle.