Currently, we scrape all details of a class from a set of endpoints, including ones for description, prerequisites, corequisites, crosslists, and so on. Attributes are also scraped this way as well, and what all of these endpoints have in common is that data is presented in plain HTML.
Upon closer inspection of the JSON data returned by the class search endpoint, attribute data is actually embedded in the JSON, as shown below:
"sectionAttributes": [
{
"class": "net.hedtech.banner.student.schedule.SectionDegreeProgramAttributeDecorator",
"code": "DI1",
"courseReferenceNumber": "75326",
"description": "Data Intensive I",
"isZTCAttribute": false,
"termCode": "202509"
},
{
"class": "net.hedtech.banner.student.schedule.SectionDegreeProgramAttributeDecorator",
"code": "FRSH",
"courseReferenceNumber": "75326",
"description": "Introductory Level Course",
"isZTCAttribute": false,
"termCode": "202509"
}
]It would be much more efficient to take attribute data from the JSON instead of parsing HTML from the dedicated attributes endpoint as it's less error-prone and will cut down on the total number of requests somewhat.
This is assuming the attribute data in the JSON is consistent with the data from the attributes endpoint.
Currently, we scrape all details of a class from a set of endpoints, including ones for description, prerequisites, corequisites, crosslists, and so on. Attributes are also scraped this way as well, and what all of these endpoints have in common is that data is presented in plain HTML.
Upon closer inspection of the JSON data returned by the class search endpoint, attribute data is actually embedded in the JSON, as shown below:
It would be much more efficient to take attribute data from the JSON instead of parsing HTML from the dedicated attributes endpoint as it's less error-prone and will cut down on the total number of requests somewhat.
This is assuming the attribute data in the JSON is consistent with the data from the attributes endpoint.