• helvetpuli@sopuli.xyz
    link
    fedilink
    English
    arrow-up
    38
    ·
    1 day ago

    So scraping means any harvesting of data now?

    We need a new name for the fairly painful process of trying to tease meaning out of people’s unstructured HTML.

    In any case downloading a bunch of stuff from JSTOR was not scraping. And he absolutely had authorised access to that data. They took exception to the quantity, mainly.

    • tiramichu@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      9
      ·
      edit-2
      19 hours ago

      Scraping has multiple meanings.

      Web scraping is a specific type of scraping, but data via APIs or even torrents could be considered a scrape, even if that data is nicely structured.

      The commonality between them is they all have the implication that:

      • the data harvesting is automated
      • the data you harvest is not owned by you, and you don’t have explicit permission to use it
      • the scope of what you harvest is broad and not targeted at retrieving specific limited pieces of data

      Any access patterns that broadly correspond to this could be considered scraping.

      • helvetpuli@sopuli.xyz
        link
        fedilink
        English
        arrow-up
        1
        arrow-down
        1
        ·
        2 hours ago

        Sure. Fine. It means all of that now.

        But what are we going to call the difficult thing that we have to do to coax, say, unstructured event listings into reasonably structured data?

    • Nugscree@lemmy.world
      link
      fedilink
      English
      arrow-up
      4
      ·
      1 day ago

      Also that he was planning, or already was, sharing the data for free. They made an example out of him because he believed such data should be for everybody.