Who has used Dropbox with a large number of files?
I recently added 761,765 files of Android source code to my Dropbox (accepting a shared folder from a friend). This is only 60 GB (out of a 1 TB quota), but Dropbox has been unable to push any of my own new files up to the cloud. Currently it says “Indexing 463,243 files” and seems to be stuck there (though the number of files fluctuates so I know that it is still running).
Did I launch a denial of service attack on myself?
Is the android codebase completely out of control? I wonder what all that crap does?
Hmm… I poked around with WinDirStat. It seems that a lot of the GBs are due to .git subdirectories (but these don’t account for a huge number of files). There are also libraries in there that predate Android. Still, American Airlines and IBM were able to support more than one call per second with SABRE on two IBM 7090 computers (see http://www.computerworld.com/article/2564361/it-project-management/sidebar–sabre-timeline.html ). I’m going to bet that the original SABRE team would have been able to give us an equally functional smartphone with about 1/1000th the amount of software!
The true size is closer to 21 GB, I think. The IBM 7090 had 32K words, each 36 bits. So that is roughly 150,000 8-bit bytes. The Android source thus takes up 140,000 times the capacity of the 7090 mainframe.
(See http://computer-history.info/Page4.dir/pages/IBM.7090.dir/index.html )
That’s why we have economic stimulus packages. If it actually had to work, there would be a recession.
Uh, oh somebody mentioned the 7090. I actually, albeit briefly, programmed it — in FAP: Fortran Assembly Program.
sounds like 1 TB Dropbox = Unlimited Wireless as to truth in advertising.
My first 7090 program was a 1 card loader in machine code created with an IBM 010 keypunch. Google that device to see what it was. The program was read as two instructions per horizontal row in the first 72 columns. That gave you a maximum of 24 instructions to read the rest of the deck of cards. An assembler and Fortran were a big step up.
There are simulators for the 7090 that you can run it on a PC.
You should be able to go to the dropbox app on your computer and use the “selective sync” section in settings to unselect that new huge folder. And then turn that folder back on selectively (at night?) to work on the huge folder.
Also useful when travelling and using a cell phone hotspot … preventing dropbox from using up all of your plan data.
Dropbox is “indexing” the files by comparing their contents to everything else on their service. If the files hash to the same thing as another file they already have, they link them, instead of copying the file, and thereby save the space. As your example illustrates, it’s a computatively expensive operation. I’m sure it’s a wonderful thing for their storage costs, but it’s a privacy issue. They don’t use encryption for exactly this reason, and it leaves them vulnerable to any request the government might make of someone’s data.
It finally went over to a green “up to date” status after about five days and everything seems to have uploaded.
I can’t wait to see what your rewrite of Android has in store!
On some systems, Dropbox performance can drop dramatically when there are over 300,000 files. This isn’t a hard limit, but it is a known issue effecting some users.
https://www.dropbox.com/help/space/file-storage-limit