I think I officially have a hoarding problem…

  • billwashere@lemmy.worldEnglish
    14·
    19 hours ago

    One think I miss from reddit… /r/datahoarder

    These were my people. I probably have 100TB but it certainly isn’t in my home directory. I’m not sure if I should be immpressed or freightened.

    • bitjunkie@lemmy.worldEnglish
      15·
      13 hours ago

      Someone may eventually write CLI programs called hast and mich that you can somehow usefully pipe to it

    • Ephera@lemmy.mlEnglish
      8·
      14 hours ago

      df only shows partitions, whereas du adds up the file sizes in the directory you specify.

      So, in particular, if you want to find out what’s taking up so much space, you can repeatedly run du -sh * and cd into the largest directory.

  • justme@lemmy.dbzer0.comEnglish
    11·
    1 day ago

    never saw the s argument and was curious what’s the difference to d. man pages are way ahead of me ^^

    --max-depth=0 is the same as --summarize

    • ShredderFeederOPAEnglish
      4·
      23 hours ago

      So just did a couple of experiments…

      sudo su -sh /home - returns permission denied errors on certain NAS subdirectories, but not a lot.

      du -sh /home --summarize -returns the same errors.

      du -sh --max-depth=0 - returns the same errors plus an error saying that using --max-depth=0 is the same as --summarize.

      ;-) for the purposes of what I was doing (creating a clip for posting) redirecting stderr to null was the best option.

      But I learned a few things today, which is cool. ;-)

      • justme@lemmy.dbzer0.comEnglish
        2·
        22 hours ago

        But I learned a few things today, which is cool. ;-)

        that is always the most important. thanks for sharing!

  • Joelk111@lemmy.worldEnglish
    3·
    19 hours ago

    I create videos, and back up all of my raw footage. I make weekly videos, and the size ranges from 50GB up to 500GB or more. I have 105TB available, 90TB used at the moment. I also have a fully redundant set of another 105TB. My employer has unfortunately made it very easy to justify hoarding, as they’ll sell me reputable used commercial drives for $10/TB.

    The video archives are 53TB

    TubeArchivist is 19TB

    Legally acquired movies and TV is 10TB

    Immich is 2TB

    Those are the main users of data. A bunch of other folders are using anywhere from a gig to 500GB, but those are basically rounding errors.

      • Joelk111@lemmy.worldEnglish
        1·
        5 hours ago

        I don’t need a raw video older than a day after the video is finished. I also don’t need TBs of legally acquired content. I also don’t need TBs of archived YouTube videos. I don’t really need a NAS. I don’t need a phone. We don’t really need any of this tech stuff actually.

        To answer your question more seriously, they’re nice to have sometimes, instead of having to re-edit finished videos to make a compilation or something. I’m also hoping that maybe I’ll see some success some day and be able to hire a more skilled editor (or even just me from the future with improved skills) to turn old media into feature-length films or just better versions of what I released. I dunno. It’s data hoarding, but when I do want it, it’s super nice to have.

        I’ve also seen multiple professional creators talk about regretting not keeping the original footage from their old videos, so I’m not making that mistake. It’s just nice to have it if you ever do want it, and I have the skills to archive it myself on the cheap, as compared to paying something like BackBlaze $7/TB/mo.

        Just checked and my average video this year is currently at 400GB, so that’s $4*2 for redundancy, so $8 per week aka $35/mo. If it were in BackBlaze, my subscription fee would go up by $12 per month forever, and it’d currently be at $630/mo, with zero redundancy. When I frame it like that, I’d be a fool not to do it!

    • ShredderFeederOPAEnglish
      1·
      10 hours ago

      I have 120TB in a dell T630 at a condo i rent on the other side of the state… I replicate some Truenas volumes and proxmox backups to over VPN…an “in case the house burns down” kind of thing…

      • Joelk111@lemmy.worldEnglish
        1·
        10 hours ago

        Yeah, I’ve considered doing that, but most of my media is static and doesn’t change and, with my upload speeds, would take literal days to sync. So I just have a set of HDDs that I keep across the state. I’ll loose some if everything fails, but at least I’ll have most of it.

        • ShredderFeederOPAEnglish
          1·
          9 hours ago

          I brought the two into the same room for the initial sync…then drove the T630 to the condo and hooked up, and it’s all incremental updates over the 1G VPN. :)

          • Joelk111@lemmy.worldEnglish
            2·
            9 hours ago

            Yeah, I have 40Mbps uploads at my house, that’s the best I can get. I create anywhere from 100GB to 3TB of footage per week. On the high end, it’d take an entire week to sync, sucking up all of my upload bandwidth for that time, meaning I wouldn’t even be able to upload the videos I create to YouTube/PeerTube in a reasonable amount of time. When I get faster upload speeds I’ll definitely have to build a remote NAS for closer to real time backups.

            • ShredderFeederOPAEnglish
              1·
              8 hours ago

              OOhh ouch, that sucks ass… If I didn’t have Gigabit synchronous at the source there is no way I would even attempt replciation…I’d be carrying spare drives with me every time I go down to visit my grandson. ;-)

    • ShredderFeederOPAEnglish
      2·
      17 hours ago

      Ask you’re employer if they’re hiring…or…you know…adopting. ;-)

      • Joelk111@lemmy.worldEnglish
        1·
        16 hours ago

        Are you living in or willing to move to Spokane, WA? I will say, I might be bias, but it is nice here.

        • ShredderFeederOPAEnglish
          2·
          16 hours ago

          I’m in Virginia - I love Washington state, spent some time in Issaquah a while back in the SeaTac area… But the last time we moved my wife told me in no uncertain terms “If you take another out of state job, you’re going alone.” (Too many years of travelling for work…)

          But I’m an awesome remote worker. ;-)

  • alastel@lemmy.mlEnglish
    852·
    2 days ago

    I was like “nothing wrong here” until I saw that T that my brain just refused to parse the first time.

  • ohshit604@sh.itjust.worksEnglish
    20·
    1 day ago

    Meanwhile here I am trying to upgrade my 512gb NVME drive to 2Tb while also still trying to afford car payments, rent and food. Rookie numbers on my part.

    • ShredderFeederOPAEnglish
      8·
      1 day ago

      For working I’m a backup and DR guy…the name was intended to be ironic. ;-)

    • ShredderFeederOPAEnglish
      12·
      1 day ago

      I know, I just paid $500 for a 24TB SAS drive that was $250 just over a year ago.

      • Someonelol@lemmy.dbzer0.comEnglish
        4·
        1 day ago

        That’s a good deal. Microcenter was selling a WD 20TB external HDD for $600. Didn’t end up pulling the trigger 'cause I’m going for a 2 drive Raid1 config on my janky setup and $1350 w/ taxes is way too steep.

        • ShredderFeederOPAEnglish
          2·
          23 hours ago

          SAS actually ends up being a little cheaper due to it not being as compatible with home systems… And it’s a refurb but I got lucky because smartctl showed it only had about 200 hrs of uptime reported.

    • nbailey@lemmy.caEnglish
      321·
      2 days ago

      There’s probably a bunch of permissions errors, filesystems warnings for cross-filesystem mounts or links, etc. all going to stderr. Linux output streams are a bit odd, 1 is stdout and 2 is stderr. So the command is redirecting the “noise” to null and just printing the actual command output. That would be my assessment, but OP could probably give a more correct answer…!

    • ShredderFeederOPAEnglish
      18·
      1 day ago

      2> means stderr… Keeps the “can’t access …” Out of the display.

    • ShredderFeederOPAEnglish
      4·
      1 day ago

      To be fair, all but about 2TB of that is on a NAS…but still.

  • bacon_pdp@lemmy.worldEnglish
    71·
    2 days ago

    Rookie numbers.

    Try that shit on datahoarders and see hoards measured in Petabytes

    • devaly@ani.socialEnglish
      2·
      1 day ago

      I went there once, on Reddit, and they are indeed over PB for some years now.

      I myself already have 50T and it feels like a bottomless pit

      • homik@slrpnk.netEnglish
        1·
        1 day ago

        I’m at single digit TBs and dreading that a drive breaks and I need a new one. Not that I have a lot of free space either.

        • bacon_pdp@lemmy.worldEnglish
          0·
          19 hours ago

          Well, a 256GB of storage in flash drive form can be obtained for $31 or less. So it is very possible to slowly save a copy of your data from the most important to the least. (Or to move 256GB of your least important data off at a time). The data curation community is a great resource for figuring out how to organize things to keep found things found