The problem is there are so many different things that can cause lag. Some of the causes could be differences between what users perceive or think lag is, having a large amount of apps in cache, low entropy, bugs in Android OS, certain apps being optimized for newer or specific hardware, certain apps being coded poorly, a certain kernel missing certain features such as a kernel without "-discard mount" built in, multiple different sd-card issues (i.e. low class sd cards, improper or unsupported partitioning, certain blocks becoming corrupt), the flags used when mounting partitions, corrupt system files, improper configuration of certain apps such as SetCPU, overheating which by extension can damage internal hardware, incorrect or disregard of wiping before install, lots of widgets loaded or updating on a homescreen, UI overlays such as Touchwiz, Blur, etc... built on top of the existing android UI, certain apps not being reniced within a given amount of time, ROMs having inefficient scripts or tweaks. Im sure there are more.
The amount of files stored on sd-cards shouldnt be a big factor in causing lag due to the way flash storage works. It doesnt have a spinning platter like HDDs do, so there isnt a penalty for seek times due to large amounts of stored data. There is however a penalty over time with read/writes. A lot of, if not almost all, NAND flash storage in phones is MLC which has some serious drawbacks compared to SLC, particularly in the number of read/write cycles it can support before peformance loss and eventual failure. The link below can provide more insight into the differences between the two types. Thats why there was some discussion a while back about safestrap usng the internal sd as overtime the wipes and writes to the .img files on the internal sdcard would cause it to take a performance hit and fail quicker.
Please Login or Register to see this Hidden Content
With all the different ways lag can manifest or be perceived to have been manifested its extremely difficult to point to one thing and say that is the cause.