As a Marine who got out a few years ago, I can speak from direct knowledge on Marine Corps. As for Sioux, I can speak from family history. I am telling you your assumptions on the Marine Corps are both right and wrong, as are your assumptions on the Sioux.
I am not saying loose formations are Inferior, since I know from personal experience they are much more effective then large formations.
What I am saying is the Spanish Calvary needed to be countered by the Mexica. Otumba was the first time they learned that lesson, and the last chance they got to learn to do so. A Spear formation is the best counter for a Calvary charge. The Mexica WERE defeated by mounted troops at the Battle of Otumba. The Calvary breakthrough there routed the leadership, causing the entire Mexica force to retreat. That shows quite clearly that their mix wasn't enough to break a calvary charge. Thousands of warriors couldn't stop hundreds, and in fact by then they might have been less then one hundred.
Yes, Formation development would be hindered by the personal glory aspect, but again, facing total war, most combatants adopt a total war attitude themselves. That would mean a more Disciplined Mexica. A good representation for this that I can see is allowing the player to pick troop types. I don't think the Mexica would ever develop a formation similar to the European formations, but I suspect that a crude spear formation would develop, as well as dedicated archers and probably more lethal weapons in the mix, but only after time goes by. Allowing for unit mix to be determined after time to allow the development of new technology allows for a catch-all type of situation, where any number of different things could have been researched and a different weapon mix would ensure from the Mexica. If total war is adopted, what happens? Mexica take up more lethal weapons and drop the less lethal ones. If they discover better formation what happens? They start to use people by weapon type. If they refine hit and run? More archers. Each of these are well represented by changing the troop tree to allow unit mix to be added.
As for the formula, you are assuming steel is the best thing since sliced bread, and while I respect your knowledge on how you have seen what weapons can do, I think you need to get an obsidian knife and compare it to a steel knife. I think there should be three sliders instead.
Tech Level, which determines access to guns
Metalworking, which determines ability to produce steel items like guns, swords, and armor
and Spanish Influence, which determines how much of the Spanish-style stuff they have.
A High Tech level would add a gunpowder Native but would require high metalworking or high Spanish influence to access, while a high Spanish influence changes most of the native weapons to European weapons. All of this decided on the interaction of native technology and understanding (Tech Level) Metalworking Infrastructure (Metalworking) and how much they accept the Spanish culture (Spanish Influence)
This can then be tempered with relations, so if they have low metalworking but high Spanish influence, they don't get the Spanish weapons, unless they are friendly with the Spanish, in which case they get a lesser quality mix of weapons.
These stats should be vulnerable to disease, after all, if all your smiths and scientists die off, how can you refine metal or pass on knowledge of the secret of gunpowder?