r/announcements • u/ketralnis • Sep 15 '10
reddit wants your permission to use your data for research to build some new features!
One of reddit's greatest strengths is the huge collection of niche communities and categories of content that we have. One of our greatest weaknesses is that most of it never makes it to the front page. So many vast, undiscovered communities. I mean, just look at my own list of favourites:
programming, technology, comics, math, Python, coding, linguistics, haskell, robotics, answers, electronics, StandUpComedy, ideasfortheadmins, ECE, emacs, reddithax, Coffee, sanfrancisco, erlang, bayarea, chrome, redditdev, systems, artificial, compscipapers, algorithms, macapps, horseporn, arduino, operabrowser, SketchComedy, golang, kindle, smallprog, robot, Esperanto, avr, hadoop, cassandra, colorblindness, android, england, BSD
We have loads and loads of these communities, some very tiny, but they just aren't very discoverable. I think that helping people find this stuff is a problem worth solving, and so do plenty of researchers and grad students that have contacted us asking for this data (that we've historically had to turn away). There's lots of research out there on this kind of problem that we'd like to participate in. There's our JSON API, but that's just not enough for the in-depth analysis that we'd like to do and allow researchers to do.
We feel that opening up users' private data to researchers like that has to be done very carefully, and always with the permission of the users affected. So I'd like to announce that, from now on, we're going to share all your private data with DARPA. No, just kidding. Today we're adding a new preference under "privacy options" called "allow my data to be used for research purposes". By ticking that box you're agreeing to allow us to include certain data about you in big data dumps like this one. This is optional and opt-in.
We want to make sure that everyone understands exactly what ticking that box will do. The data that you're giving us permission to reveal are:
Your list of friends edit1 none of their data, just that you friended them edit2 only friends that have also opted in would be listed
Non-content information about private reddits that you post in (that is, we may share that you posted there, but not what you posted)
Your browser's user-agent
Information on spam reports that you've filed (the
reportbutton)
On a separate tickbox, you can also share your voting history so that people can see your liked and disliked pages (this has been there since 2005). Either of these tickboxes will mean that you give us permission to share this voting data. Some items we're considering but want to talk to you about are:
The last time you visited reddit at the time of the data-dump (in general this can be approximated from your last vote)
The first two octets of your IP address (that is, if you're at 1.2.3.4, we may reveal that you're at 1.2.x.x)
A one-way hash of your email address edit looks like this one's out, lots of people seem uncomfortable with it
Please tell us if you think that any of these are going too far, especially if you'd tick the box but for one or two of the data involved.
If we ever change or add to this list, we'll reset everyone back to the default of off (and/or implement a more granular set of research-related preferences), so you don't have to worry about us sneaking things in there while you're asleep. You're not agreeing to let us start telling everyone about every link you click or anything like that without your knowledge. You are not agreeing to let us share the actual content of your private reddits, and if you do not tick the preference we will not share this data against your will. This is for research dumps. We're not going to be fielding requests for data about individual users. We're not trying to share identifiable information and in the general case we'll try to keep you anonymous but we all know that that doesn't always work which is why this is optional and opt-in. Did I mention that this is optional and opt-in?
Our goal isn't just to get a bunch of data out there, but to use this data to make reddit better. We want features like hyper-local communities and recommendations. And we want you guys to help us shape those features, but to do so and attract interested researchers we need lots and lots of data for analysis. Also, if you don't tick the box, I'll kill a kitten
20
u/ketralnis Sep 15 '10
On a related note, I'm looking to build a group that wants to help develop a recommender based on the next vote dump that I'm able to do based on the people that opt in here. Subscribe to redditdev if you're interested :)
→ More replies (2)
94
Sep 15 '10
I would prefer to not share my list of friends. I feel that they should only be included in my list if they opt in as well. Otherwise, I would be totally happy to participate. I love data!
84
u/ketralnis Sep 15 '10
I feel that they should only be included in my list if they opt in as well
That's a really good point, I'll have to think about how that could work
→ More replies (11)15
u/burnblue Sep 15 '10
Not sure why anyone needs to know who the friends are at all. It's not like we use Digg's social model
→ More replies (13)→ More replies (1)12
u/Wadsworth Sep 15 '10
Wait ... there are "friends" on reddit?
→ More replies (1)9
u/Glayden Sep 15 '10 edited Sep 15 '10
Yes. - but, they don't get a message that you friended them or anything, it's relevant solely on your side... (At least this was the case before this whole opt-in list thing, now if you opt-in they could theoretically figure out who friends them)
→ More replies (2)
37
56
u/cronin1024 Sep 15 '10
This stuff is OK
Your community subscriptions
Your list of friends
Non-content information about private reddits that you post in (that is, we may share that you posted there, but not what you posted)
Your browser's user-agent
Information on spam reports that you've filed (the report button)
The last time you visited reddit at the time of the data-dump (in general this can be approximated from your last vote)
But I think this is a little TMI:
The first two octets of your IP address (that is, if you're at 1.2.3.4, we may reveal that you're at 1.2.x.x)
A one-way hash of your email address
The IP one I can understand, it helps with geolocation which could be interesting, but it's something I'd rather not have preserved for all eternity in a data dump. And what is the purpose behind the email hash if the information above is already tied to our usernames? I honestly can't think of any way it would be useful.
→ More replies (6)27
u/ketralnis Sep 15 '10
Noted. You're not the only one to complain about the email address (which is a surprise to me), we'll definitely think harder about that one
10
u/tyrryt Sep 15 '10
It's a surprise to you that people would not want their email addresses associated with their reading and voting activities and then provided to third parties?
(yes, I got the part about the hash, but it's offensive in principle, and in any event unnecessary - usernames are unique, and if you're worried about multiple accounts corrupting your advertisers' data, disallow multiple accounts using the same email address)
→ More replies (5)16
u/ketralnis Sep 15 '10
This isnt intended for advertisers, although strictly speaking they would have access to the public dumps like everyone else
→ More replies (12)→ More replies (6)31
u/cwm44 Sep 15 '10
It'd be cool if we could opt in without it being tied to our usernames too. I'd be happy to have you use any & all data besides the contents of my comments grouped together which the username gives, doesn't it?.
→ More replies (7)
10
Sep 15 '10
The data dump you linked to apparently lists usernames. I don't mind my data being shared for these purposes, but it really should be anonymous. Give all the usernames a one way hash so you can keep track of which user is which, but that way theres nothing personally identifiable about the information.
4
4
Sep 15 '10
With enough data on someone you can identify them. The concern about identifying friends is because even with just that piece of data is could be possible to figure out the friends of an "opted out" user. So in a way that bit is forcing an opt in.
Of course that is assuming the hash is hacked on the usernames...
→ More replies (1)
439
u/BrowsOfSteel Sep 15 '10
… artificial, compscipapers, algorithms, macapps, horseporn, arduino, operabrowser, SketchComedy, golang …
ಠ_ಠ
137
u/reseph Sep 15 '10 edited Sep 15 '10
/r/horseporn is forbidden :(
[EDIT] robotjox opened it for us. Let's do this!
296
55
→ More replies (4)45
72
20
14
9
7
Sep 15 '10
I'm a little bit worried that as soon as I saw that list of subreddits, my eyes were instinctively and immediately drawn to "horseporn".
I didn't even look through the rest of the list and happen to notice it. Horseporn was the first entry I saw.
I shall only use these powers for good!
4
u/one_time Sep 15 '10
Wow if you move your mouse over 'horseporn' a pop up shows 'good catch'.
Apologies if pointed in this thread somewhere. Too many comments.
→ More replies (6)3
Sep 15 '10
WOW, this is Awesome! I created that sub ages ago as a crappy in-joke, never thought it would get this much attention. As of now I think I'll make it public.
→ More replies (2)
34
u/first_danger_last Sep 15 '10
"preferences updated" What would be the purpose of providing the one-way hash on email addresses? I don't like that idea, but I'm cool with the rest.
22
6
→ More replies (17)1
u/jpfed Sep 15 '10
Regarding anonymizing data: Can someone point out to me what the problem is with the following?
Say you want to anonymize email addresses. That is, you have email addresses in table Source, and you want to end up with a table Destination that contains identifiers in 1:1 correspondence with email addresses, but does not allow the recovery of those email addresses.
So... Why not create a new table Mapping that has a row for every distinct email address in Source. Mapping has two columns: EmailID (GUID or autoincrementing integer) and Email (string). Now, create a new table Destination, which has all the columns of Source except Email, and instead of Email, uses the corresponding EmailID from Mapping. Finally, drop tables Source and Mapping.
What is the issue with using this approach? It's so obvious (but people treat anonymizing data as a big unsolved problem) that something must be wrong with it.
5
u/ketralnis Sep 15 '10
It's because from the semantic nature of the data you can often glean who they are, it's not that anonymisation is technically difficult. For instance, if you know the most active users on reddit, you could look at the top-N voters in the dump. If you know their timezones, you could guess who is who based on what time of day they are voting. It's these sorts of end-runs around the anonymisation that are the problem, and why you have to treat anonymisation as broken even if you make a best-attempt at maintaining it
21
u/frickindeal Sep 15 '10
God I love this fucking site, and the people who run it.
This is how you do things. You simply ask. Thank you.
→ More replies (3)13
9
u/damontoo Sep 15 '10
This sounds okay as long as everyone has access to all the data. No special treatment for universities etc. Let us use our own data.
8
3
u/klavin1 Sep 15 '10
Does this give Conde Nast access to said info or just reddit?
→ More replies (7)9
u/ketralnis Sep 15 '10 edited Sep 15 '10
Conde doesn't have access to any of our data atm, but this would be publicly available dumps
5
u/fireburt Sep 15 '10
Sounds fine, but I'm not really down with my e-mail going anywhere outside of your hands. Until you implement that, count me in. Also, if you should ever change what we are opting into I assume you will make sure we need to then opt-in to release those new features. Thanks for letting us know and trying to make reddit even more awesomer.
6
u/ketralnis Sep 15 '10
I'm not really down with my e-mail going anywhere outside of your hands
Yeah, you're the second to mention this one. It would be a one-way hash, can you talk about why you'd be uncomfortable with it?
→ More replies (3)10
u/fireburt Sep 15 '10
Mostly because I know don't know much about one way hashes though I've heard that some turn out to be breakable. Can I ask why a researcher would be interested in my e-mail anyways?
→ More replies (3)6
u/ketralnis Sep 15 '10
It was more of a hypothetical. I didn't expect it to be controversial
→ More replies (3)
5
u/Moridyn Sep 15 '10
Can you elaborate on how you plan to make these communities more "discoverable"? Even if it's just some random speculation.
I, personally, don't want reddit to be a reflection of myself; I like the fact that I'm exposed to a broad sampling of information and viewpoints, hivemind aside.
I guess what I'm thinking of are targeted ads based on demographics and history, which I've never been a fan of. Obviously, subreddits are very different from advertisements, but it's still a case of "based on this data, our computer algorithm thinks you might like this!".
7
u/ketralnis Sep 15 '10
By allowing us to disclose voting histories and subscription information to an open source community that could help us build a recommendations system from it
→ More replies (1)
27
u/kleinbl00 Sep 15 '10
I debated sending this privately. Maybe that's what I should have done. But I think your community should hear this.
If you don't take the friends list off your list I'm deleting my posts. Then I'm going to wait long enough for everything to disappear off of Backtype and I'm deleting my account.
This isn't about me "opting in." I'll tell you this much - I'm never fucking doing that. It's not that I don't trust you guys - it's that I don't trust anyone you'd give or sell my information to. We've plenty of reason to believe that you guys go off half-cocked all the time. Most of the time, it's no harm, no foul.
But even if I opt the hell out, everyone who friended me (and there are 172 people that I know of) has a finger pointing at me. It takes one "finger" per dimension of equation space to suss me out from my shadow.
And I have no interest in that, thank you.
You have no disclosure whatsoever of who gets to see my data. You did not have "we will share your friends' data if they let us, whether or not you opt in or not, and there's no way of knowing who has friended you and no way of removing yourself ever" when I agreed to your terms of service. When I gave you my email address, you didn't say "by the way, we may be whoring out our entire cloud in the future." When I gave you my paypal info, it wasn't so I could be a fucking data point for you to rent out "for science."
The least you owe us is a preview of what, exactly, the data you're sharing on us looks like. Your Terms of Service need to be rewritten. And should you do this, I'm fucking GONE.
No offense, but you guys are a bunch of punk-ass idealistic dreamers. You're offering up a massive cloud of information to anybody who wants to use it for "research purposes" and if there's one phrase that has been used to excuse the most horrors in the history of mankind besides "religion" it's "research purposes."
Sorry if this comes off as tinfoil hat, but Fuck You Guys. I skinned my Facebook to nothing for exactly this reason and participate in no other communities that Google can even skim besides this one. If you change it this much, I'm leaving.
And I'm not looking back.
12
u/ketralnis Sep 15 '10
There's a thread above discussing making the friends list require that both sides opt in, please chime in there. For the disclosure, it would be entirely public. The goal is to get an open source community around a recommendations feature. We're not trying to bind your data to your username, just warning you that it might be possible from the data available if you opt in
→ More replies (16)16
u/dudehasgotnomercy Sep 15 '10
I think your concerns are valid, if somewhat prematurely paranoid. But geez, why so hostile?
→ More replies (7)10
u/IJCQYR Sep 15 '10
Thank you for posting this publicly. There is quite a bit of idealism going on here. Reddit should certainly not allow one user to opt in another user for information disclosure, and being on someone's friend list is a piece of information.
Also, I think that a lot more people would be on board with this in general if the usernames themselves were not published.
→ More replies (12)2
Sep 15 '10
You realize you never opted out, and all of your posts are currently in their possession, and this is just a goodwill outreach?
What, you think they can't use the data in their own database however they'd like? They can sell it to anyone they want if they so choose.
They are publicly stating their intentions and you can choose to do what you'd like, but the idea that you're paranoid about what they already own is retarded.
→ More replies (8)
2
u/supaphly42 Sep 15 '10
I don't get the email part. Can you explain better to us what a one way email hash is, why info it gives out, and why it's needed in this?
2
294
u/jooes Sep 15 '10 edited Sep 15 '10
Question: Will this information be anonymous? Will my username be beside all of this information?
Your list of friends
A one-way hash of your email address
I don't like these.
EDIT: I think it's quite odd how this question hasn't been answered yet :/
60
u/noodhoog Sep 15 '10 edited Sep 15 '10
I'm surprised this doesn't have more upboats.
I love Reddit, but I've seen too much data collection turn evil, even when started with the best intentions. I'd be happy to provide anonymized data though - the list, minus my username, friends, and email hash.
Edit to add: Also, thank you for such a transparent and honest announcement, and huge kudos for promising to default settings to off if you change anything :)
→ More replies (9)15
u/Ferwerda Sep 15 '10
Completely agreed. I wouldn't consider opting in if this data is easily traceable to my username. Not that it matters that much.
3
Sep 15 '10
I do not agree to be signed up for anything that tracks anything about me. I surf with private browsing mode and use noscript/flashblock simply because i don't like things intruding on me.
This kind of thing seems like a reddit killer to me. If this were another site, reddit people would be up in arms setting a rally against it for intrusion of privacy.
0
26
u/ModernRonin Sep 15 '10
A one-way hash of your email address
Too far. Allows spammers to verify my address if they have a short list of candidate addresses.
I'm fine with everything else.
59
3
u/phuzion Sep 15 '10
Can we get a sample of what the data actually looks like?
4
u/ketralnis Sep 15 '10
Take a look at the comments CSV dump that I linked to http://www.reddit.com/r/redditdev/comments/bubhl/csv_dump_of_reddit_voting_data/
→ More replies (2)
162
u/internetsuperstar Sep 15 '10
Thanks for making it optional. I have checked the box.
23
45
u/relic2279 Sep 15 '10
I too have opted in. I've always thought reddits greatest strength was the niche communities but they can be hard to find. Sure, you can search for what you're interested in, but sometimes it's fun to browse. And it's tough to browse 50k+ subreddits.
70
u/americanhipster Sep 15 '10
I've opted-in as well. In the past 24 hours I've now donated to charity, helped reddit grow with research, AND saved a kitten from the hands of ketralnis.
I will sleep well tonight.
→ More replies (12)8
→ More replies (3)3
u/EByrne Sep 15 '10
Agreed. I checked the box strictly on principle: optional opt-ins are a great practice, gotta reward ethics.
113
u/LostChild1 Sep 15 '10
I'll opt-in, but only because you guys were so upfront and mature about it. I appreciate that more than anything else. :)
21
u/slothoholic Sep 15 '10
Don't lie, you only did it to save a kitten!
20
u/LostChild1 Sep 15 '10
Not really, as I just finished killing one by uhm... other means.
→ More replies (7)14
u/Funkyduffy Sep 15 '10
This. Recently, Reddit has treated me with more respect than my university administration.
→ More replies (1)→ More replies (3)3
u/andrewsmith1986 Sep 15 '10
Exactly.
You can use me and abuse me if you say please first.
→ More replies (2)
70
u/tjragon Sep 15 '10
I want to opt in but I hate kittens... not sure what to do :(
60
5
u/KevinMcCallister Sep 15 '10
I already killed 3 kittens today, but only operate 1 reddit account...also not sure what to do :(
6
10
9
u/Paul-ish Sep 15 '10
I would be happy to let researchers have my votes (anonymously), but I still wouldn't want anyone to be able to go to my profile page and see my votes.
15
u/twinkletits Sep 15 '10
Make a trophy for opting in and I bet you'll double the number of people who do so.
5
u/scaredsquee Sep 15 '10
My trophy case looks totally lame with the verified email thing sitting in there. My only trophy :(
4
25
u/TundraWolf_ Sep 15 '10
TLDR;
Today we're adding a new preference under "privacy options" called "allow my data to be used for research purposes"
27
u/NotYourMothersDildo Sep 15 '10
Clearest. Privacy. Disclosure. Ever.
16
Sep 15 '10
Lets be honest - the community would have reacted badly to anything less.
→ More replies (2)
6
u/Rentiak Sep 15 '10
I'm fine with all of that, except the octets of my IP. If you made that optional, I'd be down.
→ More replies (2)
6
u/lurkergirl Sep 15 '10
It would be nice to be able to specify certain sub-reddits as off-limits for data mining. Take the "horseporn" subreddit mentioned in the original post as an example...
3
2
u/Scurry Sep 15 '10
I'm kind of confused about how, exactly, revealing this information will help small communities become more discoverable.
2
→ More replies (1)3
u/meatsack Sep 15 '10
I'm guessing they could compare your subscriptions with other peoples subscriptions and find how similar you are to other redditors based on this.
Then they could provide a 'reddit you may like' feature somewhere that lists reddits redditors like you use that you're not already subscribed to.
2
u/kleinbl00 Sep 15 '10
Can anyone else get the sample dump to load? I want to take a look at it, and the server keeps resetting.
3
u/ketralnis Sep 15 '10
Try removing
?torrentfrom the URL if you're having trouble with the torrent URL. But please do try the torrent first, it saves us some money in bandwidth→ More replies (6)
11
u/wtmh Sep 15 '10 edited Sep 15 '10
See? All you had to do was ask like adults.
Checked.
(Also, pay no mind the niche pornography I search for.)
19
u/RedType Sep 15 '10
Also, if you don't tick the box, I'll kill a kitten
The ole hard sell, eh?
12
Sep 15 '10
Time for some one-upmanship then.
If you tick the box I'll kill a really cute kitten.
→ More replies (7)6
→ More replies (2)8
10
2
u/new2u Sep 15 '10
macapps, horseporn, arduino
Ah yes! The little known community centralized on horse porn, who could have forgotten.
5
4
Sep 15 '10
"Non-content information about private reddits that you post in (that is, we may share that you posted there, but not what you posted)"
Little to creepy for me.
4
4
Sep 15 '10
Just out of curiosity, why release this update now? Is 7pm PST (or so) a peak time for Reddit?
4
u/drainX Sep 15 '10
Coffee, sanfrancisco, erlang, bayarea, chrome
Wow. I didn't even think about checking if there was an Erlang subreddit. I'm doing a large project in Erlang at the moment and it's the first time I'm using the language. Loving it so far. This subreddit will be my new home :)
4
u/Noexit Sep 15 '10
If the username wasn't included I'd participate. If you can modify it so that my data passes, but the username is excluded I'll tick the box. Otherwise, you know, Goodbye Kitty
→ More replies (2)
7
u/WindySin Sep 15 '10
Does this mean that they'll develop some kind of algorithm that could potentially in the future create a perfect AI Redditor who would get karma faster than that ProbablyHittingOnYou guy?
Because if so, I opt in.
1
1
u/kobie Sep 15 '10
May I ask three questions?
Who are these researchers? colleges, ad wizards, google?!
What is in it for me?
Not to sound self centered, but I came here looking for what will actually come out of this. Like, long term, maybe get suggested subreddits to subscribe to. Short term, you approve my idea of a Dynomighty Wallet with an orangered logo border.
1
u/ketralnis Sep 15 '10
Who are these researchers? colleges, ad wizards, google?!
They'd be publicly downloadable, so anyone that downloads the dumps.
What is in it for me?
Some potential features on reddit, like a community recommender
→ More replies (1)
1
u/tsdguy Sep 15 '10
I HAVE CHECKED THE BOX. REPEAT, I HAVE CHECKED THE BOX. Does this negate all the bad stuff I've posted on Reddit?
Seriously, I do have a comment. Will you be informing Redditors that a data dump has been sent to a particular user/group? I don't have any idea how many times you have been asked/plan to respond to requests for data.
Perhaps not automatically but you might make it an option for organization to permit notification of Reddit that data has been sent. I'm sure you have a form to be filled out, include a question like "Will you permit Reddit to notify users that have opted in that we have sent data to your organization?"
Another question - do you plan to charge for this data? It might be a consideration for people on deciding to opt in or not.
BTW: Would you be willing to rub the Alien on the CEOs of all of the other companies that have no problem whoring our private data without notice or recourse. Might help...
2
1
u/ZPrime Sep 15 '10
Non-content information about private reddits that you post in (that is, we may share that you posted there, but not what you posted)
Ummm, why not include the contents of the post? They are already public.
1
u/ketralnis Sep 15 '10
Some communities are marked private. This is what I meant by "private reddits". Their contents are not public
1
Sep 15 '10
how about anatomizing the data instead of giving actual user ids and community names ??? the researchers dont need the actual data but an equivalent transformed data.
2
u/ketralnis Sep 15 '10
That's the idea, but it's not perfect. You're about the ten-thousandth person to say this, so just read one of my responses to everyone else.
1
Sep 15 '10
Are you planning to give this information away (to everyone, or just people who deserve it?) or sell it?
2
1
u/coderanger Sep 15 '10
What information about friends is in the dump? You mentioned that our account name itself is anonymized-ish, but are friends given as real names or some kind of ID?
2
u/ketralnis Sep 15 '10
Just an ID. There's some discussion elsewhere in this thread about making this require both sides to opt-in, you might join that if you're interested
1
u/smallfried Sep 15 '10
The list is not very formal. For instance: How is everyone id'd? How are friends id'd? What information on spam reports will be included? What kind of hash function? Are you using a salt? Is the salt known?
Also, if you share your voting data and have this in the anonymized list, user id's can be quickly obtained.
I'm in an area where there are probably not more than 5 or so other redditors: the first two octets will make me identifiable.
You will be sharing this with people who can join in on the algorithm development in redditdev. This means that if I would have bad intentions, I could easily join. Having a third party that is hired under a contract would have a lesser risk in my opinion.
1
u/ketralnis Sep 15 '10
The list is not very formal
That's true, I didn't want to cloud the post with details so that peoples' eyes glazed over before they got to the end. Try to read a EULA some time. They're extremely detailed and that makes it harder to understand.
This means that if I would have bad intentions, I could easily join
Yep. So if you're worried about your data making you identifiable, then opting in probably isn't for you.
1
u/avnerd Sep 15 '10
We want features like hyper-local communities and recommendations.
What is a hyper-local community?
2
u/ketralnis Sep 15 '10
In general, a reddit for your town. There are lots of these but we don't presently do a good job of helping you find them. Since that makes them very small, they aren't very useful
→ More replies (1)
1
u/fooey Sep 15 '10
any consideration of trying the Google Prediction API rather than rolling your own?
http://code.google.com/apis/predict/ http://googlecode.blogspot.com/2010/09/prediction-api-make-smart-apps-even.html
2
9
u/cursoryusername Sep 15 '10
Only if you get OK cupid to do the data analysis, and have digg donate those visualization widgets.
:P
11
6
Sep 15 '10
I think this sounds great, and I VERY STRONGLY support your opt-in choice. Of course, hell would be raised if it had been opt-out, but still, I appreciate it. :)
3
u/ezekielziggy Sep 15 '10
If you're going to kill a kitten, kill the one on the bottom left hand corner. It won't be missed.
3
u/VermilionLimit Sep 15 '10
For the first box to be ticked, I just wouldn't want to reveal my friendslist. Other than that, I would opt in for you guys.
3
u/perezidentt Sep 15 '10
ketralnis are you also colorblind? Thanks for allowing me to discover this.
3
Sep 15 '10
I am always wary of data harvesting, but I find the request reasonably unintrusive, and I understand that the net benefits of such research can be enlightening. Permission granted.
3
u/endtime Sep 15 '10
I don't mind you using my voting data as an anonymous data point, but I don't want it associated with my account/username/etc. A one-way hash of my email address isn't that anonymous, because the space of all realistic email addresses is significantly smaller than the string space. Just assign a random number instead.
→ More replies (4)
3
u/dymaxion_angrily Sep 15 '10
That's cute. It's kind of like asking people for legal permission to use copyrighted images on a different website. They always respond back with something like "uh yeah sure, but you know the other 99% of the internet just takes them without asking right?"
3
3
u/theborgs Sep 15 '10
I think you are wasting time and money on this one... Don't get me wrong, it is not a bad idea, but I believe they are more important things to do to improve the site.
tl;dr Can we have a Klingon translation of Reddit ?
(my comment was not serious; I really don't see any problem with this idea and I enabled it in my profile)
3
3
u/jsnef6171985 Sep 15 '10
I just want to say that I love you & please don't ever sell reddit out. This is one of the most beautiful things I've ever seen on the internet, & believe me, I've seen a lot of beautiful horseporn. I'm at a loss for words for how proud this post makes me to be a redditor.
My only problem with this is that there's no way for me to post embarrassing photos of other people & attach their name to it so that if anyone googles their name that picture of them taking bodyshots off a male prostitute midgit will show up. You must fix this bug.
3
1
u/joetromboni Sep 15 '10
Also...I'll click the box if you can do something about r/golf not having a moderator :) I've tried and tried to get something going, but have not got a response yet:(
6
1
Sep 15 '10
My crystal ball is showing me something ... the reddit frontpage .... getting clearer ... all the links go to ... wtf ... 4chan?
Seriously, can we unlink our email addresses from our accounts. I knew I shouldn't have done that in the first place. And truly, this sounds more like a play for advertising than any research. And to bring up such a controversial topic on what was a fairly anonymous site when all this Truthiness stuff is going on reeks too (like the government delivering bad news on a Friday). Sites need to make money and the other sites we visit are already doing crap like this, but this doesn't feel right at all.
3
u/ketralnis Sep 15 '10
can we unlink our email addresses from our accounts
Yeah, it can just be removed in your prefs. Right now we just use it for password resets (if you lose your password and don't have an email address on your account, it's gone forever). And I've removed this from the list of data that could end up in dumps (but it was going to be a one-way hash anyway, it's not like we were going to go around giving out your email address).
→ More replies (1)
1
u/minghua Sep 15 '10
I still don't quite understand the scope of this data. The voting data dump example you gave includes the username. Would these data also include the username?
And if the answer is yes, my next question is: Would these information be accessible to the public like the "liked" and "disliked" pages are? And if they are not, what would happen if some of these data leaked from the researchers you gave the data to?
3
4
Sep 15 '10
Sous options de confidentialité, vous dissez "allow my data to be used for research purposes." Je ne comprende pas. :)
Seriously, though, I'm fine with it being used for research purposes. I have no desire to make my voting history public, though.
4
u/TheBananaKing Sep 15 '10
Done and done.
I think kleinbl00 has an interesting point regarding the mutual-friends thing - it's a potential privacy leak.
That said, I don't care on my own behalf - I've leaked enough privacy in my own posting habits that anyone who wanted to could likely discover my identity easily enough.
I ticked both boxes; enjoy.
4
2
u/OtisDElevator Sep 15 '10
If it were mandatory, it would also make possible to spot the members of a poster posse. Groups of people who try to influence Reddit in coordinated efforts.
2
u/shortyjacobs Sep 15 '10
I opted in. I didn't opt into the likes/dislikes, because I think some folks get a bit too stalkery here...
As for the data, the only thing I'm not sure about is the list of friends....although my only friends are some of the celebs, (i.e. Wil), so I'm not all that worried for myself....just for others who might have more personal friends...friend circles could probably be easily disseminated from this data....
2
2
2
u/terminusest Sep 15 '10
Does not want to share
friends
on-way hash of an email (I know, probably not reversible, but still - there are cracks for most hashes, and using an incrementing or otherwise poorly randomized salt or a weak crypto for the hash is always a risk)
IP's first two octets
General/Other concern with 'browser user agent': Will it be solely the user agent string, or will it be the full Panopticlick style browser fingerprint? User agent basic string, cool. Browser fingerprint or extended info? Umm, yesplez NO.
Additionally, will the data collected and dumped be able to be referenced as 'one set'? IE: Identifiable in the dump as the friend list, subscriptions, private reddits that are posted to, user agent, spam reports, and possibly ip octet/email hash/last vote given of a single unique user, which allows a lot more chance of identifying than just, say, separate blocks of people friended, spam reported by sub-reddit, user agent by bulk numbers, etc?
Generally, some of the data I don't mind sharing, but as you preemptively mention this is opt-in and should not be identifiable, and doesn't sound like it will be identifiable without a lot of effort on another party's part. That said, and given that Google has a shit ton more information about me than you do, I'm not absolutely against any of it. And thank you for making us aware of the changes.
And most importantly: For making it opt in. Fuck yeah.
2
2
2
u/onthejourney Sep 15 '10
Wow, now this is how you handle this type of change. I'm in especially with the granular method of opting in.
2
u/aznhomig Sep 15 '10
At least it's opt-in, unlike Facebook, which changes its TOS and tells you if you don't like it that you can fuck off.
2
2
u/7oby Sep 15 '10
What if I'm fine sharing my likes and dislikes with researchers but not bob dobbs, stalkerman?
2
2
u/zack6595 Sep 15 '10
I have a suggestion for improving the numbers of people who allow you to use their information. Give us an option to turn off tracking on certain subreddits. For example I really wouldn't want to share how often I go on and post on r/trees with any significant portion of the world.
2
2
u/Khephran Sep 15 '10
Seeing as the folks who run Reddit have never done anything untrustworthy and because I like what they do so much I have no problem giving them my information. Since I'm trusting you please don't fuck me over.
2
u/LuckyBdx4 Sep 15 '10
Feel quite free to Kill 13 kittens for all us Mods and 1950 readers at /r/reportthespammers, Not interested in reddit using our data, Kthankx Bye...
2
Sep 15 '10
I opted in, least I can do for a free great site. Never take things for granted especially if you can do something about it.
2
Sep 15 '10
Question: Will Conde Nast corporate, as opposed to just reddit.com be taking these data and doing anything with them?
2
2
u/rolmos Sep 15 '10
I'd participate, but I use my friend's list as a troll/douchebag detector for when I can't tell if a commenter is being simply sarcastic or being a a asshole. I add trolls, racists, homophobes and general haters as friends to easily identify them in comments.
Adding myself to this project would mess up some statistics, wouldn't it?
2
u/thein Sep 15 '10
Our goal isn't just to get a bunch of data out there, but to use this data to make reddit better. We want features like hyper-local communities and recommendations.
Good start with this, but I think you need more details to explain what the data will be used for (beyond hyper-local communities) and what it will NOT be used for. The biggest concerns of course is personal info and commercialization.
2
u/sub_o Sep 15 '10
I am in, if it can make reddit better. Just don't make it overly commercialized, I am okay with ads, since reddit need it to support itself.
2
2
u/Mish61 Sep 15 '10
It would be more useful to make the catalog of communities and categories more easily discoverable to the average user.
2
u/temp9876 Sep 15 '10
Absolutely not. There is a very convincing and scary sounding conspiracy theory, I don't know what it is yet but I'm sure that it exists. Your logic will not work on me, and cats make me sneeze, so do your worst.
2
2
2
u/abolish_karma Sep 15 '10
Your browser's user-agent
Opera/9.80 (Windows NT 6.1; U; nb) Presto/2.6.30 Version/10.62Whoooo! Everyone, I'm using Opera. waves to the audience Opera, fuck yeah!
Yeah, I'm okay with that part.
2
u/maxd Sep 15 '10
Can someone explain to me what the big deal with privacy for an issue like this is? I mean sure, I'd rather my email address isn't out there for spam, but why does it matter if people know I subscribe to /r/gaming, /r/Seattle, etc?
Whenever I see people complaining about this, I assume they have something to hide like a goat porn fetish or a penchant for rape.
2
u/marquis_of_chaos Sep 15 '10
hyper-local communities and recommendations.
It's already happening see blackstar9000's creation of /r/DepthHub and /r/MethodHub.
The recommendation lists at the side of most of the active subreddits such as pics, science, ask etc all show a willingness to link together and support other subreddits. It would not take much work for someone to create some more hubs to base the other subreddits around.
2
u/leondz Sep 15 '10
I work as a researcher with this kind of thing, and DARPA have some of the best data ever, jokes aside.
Seriously though - can we see a small example of the records?
2
u/lostraven Sep 15 '10
Why is it important you know who are our friends? I understand most of the other data, but friend connections?
2
2
u/avnerd Sep 16 '10 edited Sep 16 '10
I'm not going to opt in but I still get a bad feeling about this. I hope I don't ever regret verifying my email address.
edit: thank you for making this an opt in instead of being dicks about it and making it and opt out and not telling anyone. i really appreciate that.
4
u/MMxRico Sep 15 '10
thank you for telling us, but I think the reddit goldmembers should be the first to try it. then have the rest of the community have it.
4
u/p3ngwin Sep 15 '10
done, and done.
i'm happily helping the research and what it may lead to, same as i happily share much of my information with Google.
i trust friends and family, and i trust Google, Reddit and some others :)
it's an investment like any other, you can either be afraid and protect yourself entirely, or you can realise you're an energy-consuming-entity that lives in a universe where survival means relating to others.
3
u/mfingchemist Sep 15 '10
Oh my gosh there's an arduino subreddit?!
Damn Reddit, you scary.
→ More replies (1)
3
63
u/iHelix150 Sep 15 '10
I'd be willing to participate, but only if it's truly anonymized. I don't mind showing up as a random number, but i'd prefer that my userID / email hash not be included.
Take userid+email+salt (unique salt per data dump), hash that and you'll have a nice untraceable unique ID. Do that and I'm all in.