Rebuilding GitHub Star History Without Exposing User Identities
GitHub optimizes API performance by replacing identity-based stargazer lists with aggregated daily count endpoints.
1The Challenge of Star History
Maintainers use star history to measure the impact of releases or project launches on community growth. Previously, tools reconstructed these curves by scraping paginated stargazer lists from the GitHub API. This design forced tools to download individual user identities, creating significant privacy risks and inefficient database processing for popular repositories.
2Security Risks of Identity Exposure
The original stargazer endpoint returned full lists of users who starred a repository. Bad actors exploited this by scraping public lists to fuel spam and phishing campaigns. To mitigate these risks, GitHub restricted access to these lists, limiting visibility to repository administrators and collaborators to protect the broader developer community.
3Performance Bottlenecks in API Design
The old endpoint was never designed as a historical event log. For a repository like react/react with 251,000 stars, tools required over 2,500 requests to build a full curve. Each request forced the database to count thousands of entries, leading to frequent timeouts and hitting strict API rate limits.
4Database Load and Query Costs
A bug in the pagination implementation caused the database to count all existing stars just to return a single page of 100 results. In the week before access was restricted, nearly 37,000 of these queries were logged as slow, with over 1,000 being killed by database safeguards to prevent system instability.
5Collaborative Solution Development
After restricting access, GitHub collaborated with maintainers and tool builders to identify their actual needs. They discovered that developers did not require usernames; they only needed the growth curve. This insight allowed GitHub to pivot from returning individual identities to providing a purpose-built endpoint that delivers aggregated daily star counts.
6New Aggregate API Endpoint
The new endpoint provides up to 30 weeks of daily star counts in a single request. By removing identity data from the response, the system significantly reduces the volume of data processed. This design ensures that the API remains cheap to serve while providing the exact resolution required for historical analysis.
7Efficient Data Filtering Mechanism
To maintain high performance, the new endpoint filters stars by time before performing any counting operations. This architectural change prevents the database from scanning the entire history of a repository. By focusing only on the requested timeframe, the system avoids the heavy computational overhead that plagued the previous paginated stargazer implementation.
8Impact of Access Restrictions
Restricting stargazer access to repository administrators had an immediate positive effect on database health. Following the change, the database served approximately 60% fewer stargazer requests. Furthermore, the total time spent on expensive count queries dropped by about 80%, demonstrating the efficiency gains achieved by moving away from identity-based data retrieval.
9Developer Tooling Continuity
The API transition initially disrupted existing tools that relied on the old stargazer response format. However, by working directly with maintainers like those from star-history.com, GitHub ensured that the new aggregate endpoint met the functional requirements of the community. This collaboration turned a difficult technical situation into a more sustainable, performant solution.
10Redefining the API Contract
The project highlights the importance of separating data requirements from API design. The original endpoint was built to answer 'who starred this,' but it was being used to answer 'how many stars were added.' By aligning the API contract with the actual use case, GitHub improved both security and system scalability.
11Lessons in Scalable Engineering
This transition demonstrates that returning less data is not always enough to reduce query costs. True efficiency requires optimizing how the database processes information before it is returned. By filtering data at the source, GitHub successfully reduced the computational burden while still providing the necessary insights for developers to track growth.
12Future of Repository Analytics
Moving forward, the new endpoint sets a standard for how historical data should be exposed on large platforms. By prioritizing privacy and performance through aggregation, GitHub provides a model for other services. This shift ensures that developers can continue to analyze project trajectories without compromising user security or platform stability.
Original article: The GitHub Blog — Rebuilding GitHub Star History Without Exposing User Identities