YouTube Data API
Astro utilizes the YouTube Data API for data collection. In order to use the API, you need to create an API key.
The steps I used are:
- Login to Google Developers Console.
- Create a new project.
- On the new project dashboard, click Enable APIs and Services.
- In the library, navigate to YouTube Data API v3 under YouTube APIs.
- Click the YouTube Data API v3 tile, and enable the API on the next screen.
- Create credentials.
- A screen will appear with the API key.
Helpful information about the API.
The YouTube Data API works on a quota system, restricting API usage for each user per 24 hour period in order to protect service quality. In a 24 hour period, an API user is not allowed 10,000 quotas. Each API call has a different quota cost, which can be viewed here. Astro is currently only performing read (or 'list') operations, which are the cheapest API calls, costing just 1 quota each. Therefore, it is unlikely that data collection with Astro with ever meet/exceed this quota limit.
The commentCount field returned in the API response for video metadata is not always accurate.
I noticed this while working on adding a progress bar for downloading comments from a video, since I was planning to use the comment count as the 'width' or 'step count' of the progress bar (250 comments = 250 steps on the progress bar, reading 50 comments means advancing the progress bar 50 steps). When running some tests against a video with 336 comments, I noticed that my code would download 330 and then stop, leaving my progress bar to wait infinitely without completing. Thinking this was a bug in my code, I checked the dataframe constructed from the comment data, and it reflected 330.
Some online reading indicated that this may be because the mysterious 6 missing comments had some kind of moderative action taken against them. This could include:
- Filtering rules
- The video's author has filtering rules in place which block comments containing certain words/strings. When these comments get filtered out, then become invisible to every other user besides the comment author.
- YouTube moderation
- The comment was detected/flagged by YouTube's automated moderation service, restricting visibility of the comment.
- If the video's author has blocked the user attempting to comment, it seems said comment would still be included in the overall count, but its visibility is restricted.
Interestingly, not only can the commentCount report more comments than are actually visible, it can apparently return less as well:
The output above indicates that the video metadata reported by YouTube says there are 198 comments, but I analyzed the sentiment of 199 comments. Looking at the comment section on the webpage, I can see the video author pinned their own comment at the top (which was a discount advertisement). Based on this, I suspect that the commentCount does not include comments from the video author, themselves.
I also suspect that the reasoning behind this is that YouTube, in quantifying engagement on its platform, does not count the video authors amongst their pool of 'viewers', which I think makes sense. What is becoming frustrating, however, is how little I can apparently rely on the commentCount reported by YouTube to be accurate.
YouTube itself doesn't say much about how comments are counted (at least from what I could find). I found a post on YouTube Help that talks about an inconsistency in the comment count reported on the video watch page vs. YouTube Analytics, as well as another post about comment moderation and removal. Unfortunately, neither goes into much detail about how the comments are counted, so I'm left to speculate.