Curl gives html error 1020 when making HTTP request to opensea API

I'm trying to make a request to the OpenSea.io API. When I go to the network inspector I can see a whole slew of requests that come through to/from the page. When I select one, right click, and choose copy as curl I can then paste that into my terminal and normally the data comes through as output to the terminal. For a few reqeuests, I got a message about binary output that I was able to resolve by modifying the request. For example:

curl ' \ -X 'GET' \ -H 'Pragma: no-cache' \ -H 'Accept: */*' \ -H 'Accept-Language: en-US,en;q=0.9' \ -H 'Accept-Encoding: gzip, deflate, br' \ -H 'Cache-Control: no-cache' \ -H 'Origin: \ -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Safari/605.1.15' \ -H 'Connection: keep-alive' \ -H 'Referer: \ -H 'Host: api.opensea.io' \ -H 'X-API-KEY: 2f6f419a083c46de9d83ce3dbe7db601' \ -H 'X-BUILD-ID: da14c5fd3811187c88141eb116061b5f6cf87f45' 

The above gave me the binary error message, I resolve it by adding --compressed at the end to decompress the "binary" data and removed the br option from the encoding header. The below request works just fine in my terminal now.

curl ' \ -X 'GET' \ -H 'Pragma: no-cache' \ -H 'Accept: */*' \ -H 'Accept-Language: en-US,en;q=0.9' \ -H 'Accept-Encoding: gzip, deflate' \ -H 'Cache-Control: no-cache' \ -H 'Origin: \ -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Safari/605.1.15' \ -H 'Connection: keep-alive' \ -H 'Referer: \ -H 'Host: api.opensea.io' \ -H 'X-API-KEY: 2f6f419a083c46de9d83ce3dbe7db601' \ -H 'X-BUILD-ID: da14c5fd3811187c88141eb116061b5f6cf87f45' --compressed 

So that's all fine and dandy, but that didn't fix my issues for all of the of the requests. I went through and found the requests that have the data that I'm looking for but they give a new error about not being the website owner. Consider the below request:

curl ' \ -X 'POST' \ -H 'Content-Type: application/json' \ -H 'Pragma: no-cache' \ -H 'Accept: */*' \ -H 'Host: api.opensea.io' \ -H 'Cache-Control: no-cache' \ -H 'Accept-Language: en-US,en;q=0.9' \ -H 'Origin: \ -H 'Content-Length: 451' \ -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Safari/605.1.15' \ -H 'Referer: \ -H 'Accept-Encoding: gzip, deflate' \ -H 'Connection: keep-alive' \ -H 'Cookie: _ga_9VSBF2K4BX=GS1.1.1653330281.9.1.1653332997.0; csrftoken=BVdZtaJOMRxED1ALVr79hZfFHIcUUTeNokvuFbqkb17fPoZiEqpe5Fb26Mq4RQsg; sessionid=eyJzZXNzaW9uSWQiOiI0MzJjMWVlYi0zY2Q5LTQ4Y2QtODljZS1jZWFhNzk0NzI2ZDIifQ:1ntDPZ:iRgNCzJHvxP1nDBSR90Hjx4hcpPy8UmpZl7GG6lV2e8; ajs_anonymous_id=41ec97c3-3ebf-467b-a921-a31f94abeb2f; amp_ddd6ec=yUkvg9MB9AgtD0-EafL8wO...1g3p2k0km.1g3p52466.5c.54.ag; _fbp=fb.1.1652624043939.1609498506; _ga=GA1.2.337370304.1652623932; _gid=GA1.2.1049414718.1653330282; _uetsid=9d339a80dac511ec84300fb0b22c8619; _uetvid=ebc21490d88011ec99749d8ebc9bcd13; __cf_bm=OZmIijoynqXFgy9j69FEOB2a0As_1yLXG3751dUFAO4-1653332831-0-AX1rqerC9b2mttE3Lg4rIp33aWgqCGg2fozR3+cJTaeEEJ6xgpz1/VY5OIrHCONfYfGI26n0qHHCGtxb5YDwVBw=; cf_chl_2=; cf_chl_prog=; cf_clearance=mfMY41rDtGcV.Hkkmp5dZkZUtz10Y7fXRmobKhROBlw-1653331507-0-150; _gcl_au=1.1.13890619.1653330282; __os_session=eyJpZCI6IjQzMmMxZWViLTNjZDktNDhjZC04OWNlLWNlYWE3OTQ3MjZkMiJ9; __os_session.sig=xyK0HcEq8hEtOPpbnB0ra5A18qm3t-xGKx_2YDCmObc' \ -H 'x-signed-query: d73eda68d997705a2785aa8222d5a3c5663c392d0df699f665e44fb31e14642b' \ -H 'X-BUILD-ID: da14c5fd3811187c88141eb116061b5f6cf87f45' \ -H 'X-API-KEY: 2f6f419a083c46de9d83ce3dbe7db601' \ --data-binary '{"id":"TraitsDropdownQuery","query":"query TraitsDropdownQuery(\n $collection: CollectionSlug!\n) {\n collection(collection: $collection) {\n assetCount\n numericTraits {\n key\n value {\n max\n min\n }\n }\n stringTraits {\n key\n counts {\n count\n value\n }\n }\n defaultChain {\n identifier\n }\n id\n }\n}\n","variables":{"collection":"boredapeyachtclub"}}' --compressed 

When the webpage makes the request, the site server returns back a JSON file with all kinds of useful data inside. But for some reason when I make the request it gives me back an HTML file and says:

<h1> <span>Access denied</span> <span>Error code <span>1020</span></span> </h1> <div> <p>You do not have access to api.opensea.io.</p><p>The site owner may have set restrictions that prevent you from accessing the site. Contact the site owner for access or try loading the page again.</p> </div> 

Can anybody help in resolving this? What changes do I need to make to the curl request so that I actually get the JSON data I'm looking for? I understand the page is saying that I am not the website owner and that's correct, but then why does it give the JSON data to my browser and not to me through a CURL request? How does the server know the difference between my terminal and a browser making a request when I pass through all of the same headers and cookies that the browser had given it? I noticed that in the cookies there was some cf_bm and similar cookies that hold some info like a unix time stamp. I tried to pass along the current unix time stamp generating on the fly using NODE.js and Axios but I still got the same message so I believe there's something more going on besides a cookie difference. Additionally, I tried finding the cookie values from previous requests to see if maybe the server gave it some info that you have to send back later but I couldn't find any matching values between one request to the next.

Any help is much appreciated, both in fixing this specific problem as well as explaining the overall process of how the server identifies the differences between browser and terminal.

Know someone who can answer? Share a link to this question via email, Twitter, or Facebook.