r/Python • • Dec 21 '19

I wrote a Python script to generate your personal YouTube report.

Post image
1.1k Upvotes

97 comments sorted by

89

u/godblesspiggy Dec 21 '19 edited Dec 21 '19

3

u/atlantabrave10 Dec 21 '19

This is incredible. Do you by chance know if there is a way to search github for other projects like this? I have been searching for a way to find completed "templates" of projects but I have not come up with anything good.

2

u/godblesspiggy Dec 22 '19 edited Dec 22 '19

Thanks, try search by package, e.g. reportlab.

Edit: grammar

2

u/AnswerAwake Dec 22 '19 edited Dec 22 '19

So I just finished running this. It seems like my history only goes back to 2018 but I know I have been watching Youtube since at least 2007 using this account. Is this a Youtube limitation or did I maybe wipe my history and forgot about it?

EDIT: It appears that my history is split across my original Youtube account which was a username and the rest being on my Google Account. At some point Google forced users to transition their Youtube accounts into a Google account after years of giving users the option to avoid linking them which I handedly took advantage of until I could no longer do so. So downloading the data for my original account, I can see that it goes back to 2011. I'm pretty certain there is more data but I guess Google does not keep records that long? Maybe I only made an account in 2011. Interestingly it overlaps with the other account. Wish there was an easy way to merge the two accounts and generate one report.

This was an amazingly insightful script. Thank you for creating it!

2

u/godblesspiggy Dec 22 '19

Thank you for the information, I really appreciate it.

20

u/[deleted] Dec 21 '19

Oh god I'm gonna do this and it's gonna be terrible because I watch a stupid amount of YouTube

351

u/Yoghurt42 Dec 21 '19

Did you really have to use Comic Sans? I thought it was a parody for a few seconds.

123

u/godblesspiggy Dec 21 '19

Thanks for your suggestion, I am gonna change it to Arial.

37

u/Doomphx Dec 21 '19

You could be like the president of my company and only use arial bold >.<

39

u/[deleted] Dec 21 '19

Yeah make it something more mature like papyrus

17

u/normalism Dec 21 '19

YOU KNOW WHAT YOU DID!

1

u/[deleted] Dec 21 '19

Or Times New Roman.

2

u/Tydrous Dec 22 '19

I think it would look smashing in wingdings

16

u/[deleted] Dec 21 '19

Don't let the haters trick you. Comic sans is fine!

74

u/[deleted] Dec 21 '19

It really isn't

3

u/[deleted] Dec 21 '19

😦

13

u/[deleted] Dec 21 '19

f the haters, add emojis too!

-7

u/[deleted] Dec 21 '19

Sure, it's fine... if you're making a poster for a kindergarten

-9

u/bzsearch Dec 21 '19

It _realllllyyyyyyyyyy_ isn't.

1

u/Etheo Dec 21 '19

You might as well use Papyrus then.

34

u/dado_b981 Dec 21 '19

Cool. Could you make a version for PornHub?

13

u/godblesspiggy Dec 21 '19

what do you expect on the wordcloud? :D

8

u/cristianthechinch Dec 21 '19

Somebody please get on this!! Saying for a friend.... Yep.... Jokes aside, wouldn't you need a pornhub account to track it? Unless it's available in your browser history I wouldn't know how to access that information.

2

u/what_comes_after_q Dec 22 '19

Who actually has a pornhub account?

39

u/engineertee Dec 21 '19

How would someone (a beginner) even start a project like that? Do you list al the functions you think you might want to call and then start filling in the blanks? I have been reading about functional decomposition but I want to go through an actual project. This one looks really interesting.

Good job OP, one day I’ll make it to that level...hopefully :)

96

u/[deleted] Dec 21 '19

Grab a paper and :

  • write down the name of your creation : Reddit funny username finder.

  • write a general description : what your program will do ? (Not how) I want it to go on Reddit and find funny usernames automatically.

  • define every term you used : Reddit : a web site. Bot : user controlled by a program without my intervention. Funny : (tough to define what is funny). Username : a string that I can judge as funny or not.

  • specify in details your program functionality : how does it communicate with the website ? How frequently ? How does it find the usernames ? How does it judge what is funny ? How does it store them ? In other words, anticipate the problems as much as possible.

  • start implementing slow but surely. Validate every step you do: 1st let's connect to Reddit and display any submission on any subreddit. Let's select what subreddit. Let's find all the users in that sub. Let's find all their usernames. Let's find a way to pick a funny name. Let's make this bot automatic.

Some details are : to go for a functional or object oriented programming ? Is it possible with python ? How is python the best language for this ? Is there a module that does this ? How much time will I invest in this ?

I hope this guidelines will help you.

18

u/my_name_isnt_clever Dec 21 '19

This is an off topic but honest question: why are you putting a space before colons and question marks? You don't do it for periods so why for the other two? I see that sometimes and I'm curious why people do it.

21

u/Thalinde Dec 21 '19

This is how you do it in some languages other than English. For example you add these spaces in French (and not before periods or points). As someone who writes a lot in both languages, it's a pain to remember to do it correctly all the time.

8

u/[deleted] Dec 21 '19

Bingo. It comes naturally to me as I remember both but only French /German or Arabic...

4

u/WonderfulPlay Dec 21 '19

is this what system design interviews are about?

3

u/[deleted] Dec 21 '19

Yes, Real-time embedded systems, system design or any serious approach to a complex problem. Search for the V diagram.

2

u/Brandhout Dec 22 '19

All the usernames with "boob" and "PM_ME_YOUR" are funny of course, duh

1

u/insane_playzYT Dec 21 '19

And also to add onto the Reddit username idea, in Python there is the PRAW module made to interact with Reddit's API which could be useful

4

u/[deleted] Dec 22 '19

Well, I wanted to push this fella right here to find it.Thanks I guess.

2

u/[deleted] Dec 25 '19

Tremendous help. Thank you all. Keep the great pointers coming.

2

u/solderfog Dec 21 '19

You can just start studying the API for something like this. Many services expose functions, not only via web (which most of use are used to) but also through APIs. Or, like Craigslist, via RSS. There'll be documentation that you can read through so you can figure out what you're interested in. From there, you can start writing some test scripts in your favorite language. Keep working it and eventually, you end up with a useful program.

-3

u/[deleted] Dec 21 '19

The first step, if you haven't done it yet, is to read ALL of this

https://automatetheboringstuff.com/

7

u/Ph0X Dec 21 '19

I don't want to be petty, but I disagree. While that's a great book, I think reading shit ton before even getting started isn't always productive. Most of the stuff you read doesn't make too much sense without context and having been there.

I find it much for productive to jump right in, try automating pieces and then reading up when I get stuck. Also make lots of stuff and as you go read that book to enhance your knowledge. But don't gate your experimenting on finishing that whole book

4

u/[deleted] Dec 21 '19

I find it much for productive to jump right in

People learn in different ways. While that may work for you, it may not work for others.

The best way is to explore all the options and find what's better for you

3

u/engineertee Dec 21 '19

I’m half way through the Udemy videos. But so far it hasn’t covered how to you plan a full project, it is still covering the basic building blocks.

1

u/[deleted] Dec 21 '19 edited Dec 21 '19

Video training is not as good as text training.

The one I linked is an excellent guide to get started with lots of practical examples

2

u/engineertee Dec 21 '19

Ah I did not know that, I though it was the same thing. Thanks for the tip.

1

u/engineertee Dec 22 '19

I think there was a misunderstanding. I’m going through the automate the boring stuff udemy video course

1

u/Alberaan Dec 21 '19

Some time ago I was trying to do something like this. I wanted to check what video tags I watched more. Did you find any functionality for this in the API?

1

u/godblesspiggy Dec 22 '19

I didn't find any API for it, but I tried to do the trick by using web-scraping.

You can find the video tags from html source codes.

Here's an example of web-scraping version: https://i.ibb.co/6JRBLCg/Screenshot-1.png

It looks more fancy but it is time-consuming(12 videos/s)

1

u/Psychedoolic Dec 21 '19

CHIN

3

u/godblesspiggy Dec 21 '19

I was searching for thanos chin

1

u/ToolBoxTad Dec 21 '19

I was wondering what the youtube activity score is? Can you elaborate on how this is calculated and what it represents?

Edit: When I ran it using the documentation, the word cloud wouldn't install for some reason. I needed to go remove it from the requirements and the python code for the program to run. Other than that, works really well!

1

u/godblesspiggy Dec 21 '19

The youtube activity score is calculated by using the numbers from "breakdown of your activity on youtube".

The more you use YouTube, the more score you have.

You can find the function on line 138 of report.py.

1

u/buurd Dec 21 '19

When I try to run it, I get an Index error "list index out of range". Any idea why?

1

u/godblesspiggy Dec 21 '19

As mentioned on step 3, make sure to set your Google Account language to English.

1

u/buurd Dec 21 '19

It's already set to English

1

u/kparadocs Dec 22 '19

I hope I get a reply, I am unable to debug this.

2

u/godblesspiggy Dec 22 '19

Delete wordcloud from requirement.txt.

https://www.lfd.uci.edu/~gohlke/pythonlibs/#wordcloud
Try install wordcloud from the link above, cpXX is the python version you are using.

1

u/kparadocs Dec 22 '19

Thanks, will check that out

5

u/MAAltimime Dec 21 '19

Incerdible!💙👏

2

u/69shaolin69 Dec 21 '19

Wow, great

2

u/69shaolin69 Dec 21 '19

I might do something with JavaScript considering the files are in HTML probably

4

u/kaihatsusha Dec 21 '19

The Google history data is downloaded into an XML format file.

2

u/69shaolin69 Dec 21 '19

Just got my data, Now I’ll start to design UI

2

u/soap1337 Dec 21 '19

This is pretty neat! Well done! My colleague did something similar for his plex server

2

u/BoltBoy Dec 21 '19

Good job! This is actually rly cool, I’ma try it

2

u/puttestna Dec 21 '19

"Chinese rocket tutorial" :D

2

u/MrK_HS Dec 21 '19 edited Dec 21 '19

Ehm, you might want to blur some words from your searched words bubble...

2

u/MattR0se Dec 21 '19

I was expecting far more emberassing stuff tbh

2

u/BonnieCFabiano Dec 21 '19

jez how did you get to this level... probably you did this when you got bored in your free time it's like a joke for you.

2

u/Zylonity Dec 21 '19

Hey, just wondering, how far back does this go, my first watched video is definitely not accurate (2017) and I feel like it picked out a random one, my first liked video actually goes back further (2015)

2

u/[deleted] Dec 21 '19

Maybe youtube didn't save the videos you watched in 2015 but it started doing it in 2017?

2

u/[deleted] Dec 21 '19

why many terms in the bubble are missing the last char?

intervie, tutoria, basketbal, reactio

4

u/jabbeboy Dec 21 '19

Comic sans is the abomination of fonts. Please remove it 😂😂 the rest looks great

11

u/cristianthechinch Dec 21 '19

So are emojis

0

u/jabbeboy Dec 21 '19

Not as worse

1

u/roberto257 Dec 21 '19

This is really cool!

1

u/Ungreon Dec 21 '19

Badass dude. Gonna check it out

1

u/Farconion Dec 21 '19

cool, but I don't need to hate myself more with this

1

u/kyle1elyk Dec 21 '19

Pretty cool! Google does have an option to "not save" your history though so there is no search-history.html file in the Takeout which causes it to fail.

I did a 2 minute little hack to get it to work by commenting some lines and returning empty for the search terms to get a result

1

u/JimmyDelicious Dec 21 '19

Gotta watch some pytho videos...

1

u/kamil2098 Dec 21 '19

I see girl in searches... Hmm..

1

u/theguy2108 Dec 22 '19

A good and interesting project. Will do it once I finish the current one.! RemindMe 1 week

1

u/the-shit-posting-god Dec 22 '19

i knew about this project via git news [android app]

then i requested the data too

its 500 MB [only yt data]

just imagine how big google is

1

u/unchiusm Dec 22 '19

This is awesome , I just tried it today after Google finished compiling my archive . Unfortunately the first watched video from the google files only goes back to 2018.

But I went and read trough my old comments.

Boy , I had a trip down memory lane.

Very nice work , I just started learning python not long ago . I really enjoyed your script. Keep up the goodwork !

1

u/PythonForProgrammers Dec 22 '19

What is parse.py

1

u/tewojacinto Dec 23 '19

Very impressive, I'm going to use it somewhere down the line. I was wondering it can analyse a channel, I see business value there.

1

u/Miv333 Dec 24 '19

First Watched Video-- Wrong, I assume Youtube didn't collect this data that long ago? (It should be my First liked video, since I created an account to comment and like on it)

First Commented Video-- Wrong, and it's a channel (xbox? lol) not a video.

Most Watched-- Correct

First Liked-- Correct

First Searched Words-- Wrong, assume like the top, youtube didn't collect data that long ago?


Watch: 21612

Search: 5353

So I watch an average of 4 videos per search.

Like: 1410

Comment: 1331

--

Activity score 74.5, whatever that means.


If possible I have a suggestion for the word bubble... group like terms together somehow. I have 5353 searches, if most of those I only searched one time, something I searched 2 or 3 times is going to be prominent. For example, I have 'bass boost' 'bass boosted' and 'bass' all listed, but I think those could all be considered like.

Also, 'and' 'you' 'of' are all listed as individual search terms. 'My Heart' and 'Will go on' are listed separately, though it's a single search term.


Cool info graphic though. Especially the time plot.

1

u/barnabas09 Dec 31 '19

im having so much fun looking at that search wordcloud. "nigga" "wiper blade" " roblox" "rich chigga"

0

u/DaWatermeloone Dec 21 '19

Sry for being a bit picky but you mixed up 12 pm and 12 am in the usage report

4

u/godblesspiggy Dec 21 '19

I actually did a search for it, found out that 12pm for noon and 12am for midnight.

0

u/DaWatermeloone Dec 21 '19

That is totally against any sense of logic but if you say so lol

3

u/DamnFog Dec 22 '19

That's how it is. That's why 24 hour clock is the master of all.

0

u/Fraserac67 Dec 21 '19

Love to try one script with one app. Love the graphic displays. 👏