README should indicate what permissions are needed on a token

[hyandell]

Which permissions should a GitHub token be permitted?

[jeffmcaffer]

Agreed on the overall notion of explaining better how tokens are used/needed.

I am concerned a bit about maintenance. The GitHub doc covers what permissions are needed for which apis. That may change over time. Also, as we harvest more and more different types of data (i.e., calling more and more API) that list will need to be kept in sync. That and the actual set of APIs called depends on the Visitor Map you are using and the resources being queued.

So there is no easy "get a token with these permissions" answer.

One way might be to point to the code that does token resolution. That has a table of the APIs that may be called and the ideal permissions for the tokens to use. Or we can surface that list in the readme as a table and take on the maintenance burden.

Note as well that some APIs will give you more data using a token with more permissions further muddying the water.

Finally, having too few permissions is not fatal. The crawler will simply fail on those requests and they will end up in the deadletter queue. You can watch that queue and see what is failing and then supply tokens with more permissions if want that data.

[BenLloydPearson]

One way might be to point to the code that does token resolution. That has a table of the APIs that may be called and the ideal permissions for the tokens to use.

From my understanding, it seems the token needs admin ability for things like organizations. Why isn't read only access sufficient? I've granted read access to orgs for my token, but the crawler is placing org requests into deadletter.

[jeffmcaffer]

Some of the org API calls need admin permissions (or at least did in the past). What is the failure noted in the deadletter entries?

Have you tried using a token that has admin permissions? does that work?