The Union had four structured disciplined intelligence gathering operations. The South had but one, and Stuart's cavalry did not link up with Lee in time. This made a difference.
Lee's march behind South Mountain while controlling the passes was intended to hide his location and numbers. However, the Union had a spy network in the Cumberland Valley. Numerous responsible citizens had been trained to count armies. They could recognize how various sized units moved on the march, and were able to identify specific units. This information was given to young couriers, who carried it north of the Confederate controlled areas, to where the telegraph system was intact. Thus, Washington DC and General Meade had a fairly complete list of Lee's forces and locations, only a day or so out of date.
General Longstreet had hired a spy who performed a similar service for the Confederates. James Harrison moved among the Army of the Potomac as it came north. Late on Sunday June 28, he told its position to Lee, and perhaps that Meade was now in command. Lee soon began moving his entire army to the Gettysburg - Cashtown region, though his first order was for a concentration west of South Mountain at Chambersburg.
The primary job of the Union's Signal Corps was communications. To send messages, a small detachment of men with signal flags and field glasses were placed on tall hills in the army's area of operation. Thus, during daylight, messages could be relayed from hilltop to hilltop - from Gettysburg to Washington - in a matter of minutes or hours. Union response time to an order issued a distant unit was thus sometimes much faster than the Confederates, the Confederates being limited to the speed a horseman could ride.
Jeb Stuart cut the telegraph wires that had linked Meade with Washington. Some historians claim this was a vast relief and release to Meade, freeing him from War Department restrictions. Not so. The Signal Corps' hilltop flag wavers filled the gap, providing timely reports from the spies as well as directives from the War Department.
While their prime responsibility was communications, the Signal Corps could and did report what they saw of enemy activity. The Signal Corps station on Little Round Top delayed Longstreet's deployment on July 2nd, and warned Meade that a flanking movement was in progress.
The South had a much smaller Signal Corps, not nearly large enough to provide solid communications or intelligence in hostile territory.
The AoP had an organized system for interrogating prisoners. They attempted to track where prisoners were taken, how many were taken, and what unit they came from. Late on the evening of July 2nd, just after Meade and his generals had decided to stay and fight on Cemetery Ridge, the prisoner report came in. By comparing the spies' version of Lee's order of battle with the prisoner interrogations, Meade knew that every unit Lee had with him, except Pickett's division, had been engaged on July 1st and/or 2nd. With the exception of Pickett, Meade knew Lee had no more fresh troops. Upon hearing this, one Union general who had been half napping, seemingly paying little attention to the proceedings, sat up to congratulate Meade. If Lee had so few fresh troops left, the battle had already been won.
The South had no similar organization.
The other major intelligence arm was the cavalry. Buford's cavalry division, operating on the AoP's left flank, had given Meade good intelligence of Lee's position, supplementing the spy network's list of Lee's forces with an up to date account of positions. General Kilpatrick did less well on the right flank. Kilpatrick found Jeb Stuart only when a Rebel cavalry charge hit the rear of his division column.
Reports to headquarters serve two purposes. For Buford, the reports were used as a mechanism to inform Meade what was going on. To Kilpatrick, reports were a tool to achieve glory and promotions. Thus, Kilpatrick's reports contained exaggerated to fictional accounts of how the 3rd Cavalry Division had fought and defeated assorted imaginary enemy formations. This was complicated by Jeb Stuart's and Early's presence near enough Kilpatrick's area that the reports couldn't just be dismissed. Kilpatrick's fictional reports created confusion in the otherwise well structured Union intelligence gathering operations. Fortunately, Kilpatrick's style of writing reports was moderately well known. Still, Meade was slow to leave Pipe Creek, to support the I and XI corps at Gettysburg. Was the delay in supporting Gettysburg due in part to Kilpatrick's fictional threats to the Union right?
The Confederate cavalry had similar gaps in quality. Jeb Stuart had kept the best of the southern cavalry with him as he marched to link up with Early in York or Johnson in Carlisle. Collisions with the Union II Corps and Kilpatrick's cavalry delayed Stuart's march. Stuart's ride remains controversial. By one account, moving a cavalry unit 20 to 30 miles a day is pushing it. It can be done, assuming the horses were fresh as the movement started, and assuming one doesn't try to do it more than three days in a row. On the third and fourth consecutive days or such marches, the horses would be expected to break down.
Early in Stuart's march, he captured a Union wagon train. Many historians criticize Stuart for keeping this train, which supposedly slowed his march. What is seldom mentioned is that a primary cargo of this train was oats. If one grazes the horses on grass, one needs five hours of grazing a day, and one can't push the horses too hard for two hours after they graze. A wagon train full of oats allowed more time on the march. While a fully rested pure cavalry column can outpace a wagon train, Stuart's mounts were exhausted by the time they captured the oats, and were likely not slowed by the train. The combined column, with the wagon train and its oats, moved 15, 30, 22 and 26 (as the crow flies) miles in subsequent days. They fought at Westminster, Hanover, Carlisle and Gettysburg. This was a pace sufficient to kill horses. It was not a joyride.
I've an obscure what if. Suppose Buford had caught Harrison before he could report Meade's concentration at Frederick? Lee's orders to Early to rejoin the rest of the army might not have come for another day or two. Early would quite likely still be at York when Stuart and the cavalry arrived. What were Stuart's orders? Scout and screen for Early at York? If not for Harrison, Stuart would quite likely have provided first warning to Early of four Corps of Union infantry marching hard straight for York. Stuart's extremely hard marching and fighting to provide this vital information could have resulted in a decisive break for the South, allowing Early the time to rejoin the rest of the army. Alas for Stuart, Harrison had already provided this information, thus Stuart is judged as a goat, not a hero. One cannot fault Harrison for providing information in a more timely fashion, or Lee for acting on this information, but one should not blame Stuart for a failure to screen and scout for Early at York. Stuart marched hard and fought hard to stay ahead of Meade's march, to get knowledge of this march to York before it was too late.
Stuart had three choices at the start of his march. He could have marched behind Lee, competing with the infantry for limited road capacity, and marching the long way around South Mountain. He could not have traveled just east of South Mountain. The Union VIII Corps, stationed near Harper's Ferry, had been stirred active by one of Stuart's raids just before the Gettysburg campaign. The fords of the Potomac river required to pass between the AoP and South Mountain were held against Stuart. The third option, the option Stuart took, was the eastward route, passing between the AoP and Washington. It worked, but as Meade marched the Union army quickly, it did not work fast enough. Stuart had to use his horses hard four day s in a row, taking new mounts from the Maryland and Pennsylvania civilians. He thus met Lee on July 3rd with trained cavalrymen riding on plow horses and lady's palfreys.
The marches of June 29th, plus Stuart on the 30th in orange. Stuart passing between the AoP and South Mountain was not an option. Meade had the XI, and I Corps with Buford's cavalry division protecting his western flank. The VIII Corps - the Harpers Ferry garrison - was also holding the Potomac River crossings near Harpers Ferry and Point of Rocks. The eastern route was shorter, and had more open territory available to avoid opponents. Stuart cut it a little close to the quickly moving AoP, with resulting skirmishes at Westminster on the 29th and Hanover on the 30th.
Stuart's error was in leaving no margin for error, though it is not Stuart's fault alone. Lee and Longstreet's orders to Stuart clearly approved the eastern route. Regardless of who shared responsibility for the decision, the first delay encountered would be fatal, assuming one has 20 20 hindsight, and knows that the Army of the Potomac was going to change commanders and move much faster than it ever had before. Stuart and Lee were basing their planning on the AoP's standard slow marching doctrine.
"If you find that he [Hooker] is moving northward, and that two brigades can guard the Blue Ridge and take care of your rear, you can move with the other three into Maryland, and take position on General [Richard] Ewell's right, place yourself in communication with him, guard his flank, keep him informed of the enemy's movements, and collect all the supplies you can for the use of the army."
"I think your passage of the Potomac by our rear at the present will, in a measure, disclose our plans. You had better not leave us, therefore, unless you can take the route in rear of the enemy."
West of Gettysburg, on the evening of June 30, Pettigrew tried and failed to convince Pender, Hill and Lee that a veteran AoP cavalry force (Buford's division) was defending Gettysburg. None of them would believe the Yankees would move that far north so fast, challenge infantry with cavalry if they had, or that they could get to Gettysburg without Stuart giving warning. It was all unbelievable, unheard of. Pettigrew must have mistaken local militia for AoP veterans. (The next day, when the I Corps appeared on the field, there was no room for doubt. One Confederate was heard to say, "Tain't no militia - that's the Army of the Potomac!")
Yes, some of this might be 'Victory Disease,' a fatal Confederates contempt of enemy capability resulting from too many recent victories. However, part of it was the Army of the Potomac coming of age.
There were other cavalry forces available to Lee. These could have been summoned when Stuart became overdue. They had been used, however, to cover the gaps in South Mountain, and to guard his rear, important tasks well behind Lee's forward advance. When Stuart didn't report, they could not have reached Gettysburg in time to have made a difference. They were also filling significant functions, and were less well trained than Stuart's people. Lee chose not to retask his other cavalry, but to leave them to their original tasks. He chose to fight blind, but with a secure line of retreat.
Meade had a reasonably complete picture of Lee's positions and numbers. Lee had the report of James Harrison, and what he could see through field glasses from Seminary Ridge. It is said that Gettysburg was Meade's best campaign, but Lee's worst. Their intelligence operations might be the major cause. Lee was fighting blind. Meade had as good a picture of his enemy's positions as one could ask for in the days before reconnaissance aircraft.