The process variant of friction surfacing (FS) depositing multiple layers on top of each other is known as multi-layer friction surfacing (MLFS). Due to the solid-state nature of the process, re-heating is significantly reduced compared to common fusion-based AM techniques. The work gives a detailed and fundamental insight into the microstructure along the MLFS build direction for two different aluminum alloys and different process parameters. Focusing on the grain size distribution and recrystallization ratio, the stacks show a higher degree of recrystallization and finer grains at the interfaces. The observed grain sizes at the interfaces were 2.0 µm (AA5083) and 1.1 µm (AA2024), and 5.8 µm (AA5083) and 3.1 µm (AA2024) at the layer center. For the non-precipitation-hardenable alloy (AA5083), the local microstructural trend could be related to the hardness distribution along the stacks, i.e., a slightly higher hardness at the layer interfaces (95 HV) compared to the layer center (90 HV). The relationship is more complex for precipitation-hardenable alloys (AA2024), which show a rise in hardness between 40 HV0.2 and 45 HV0.2 along the stack height. The effect of subsequent layer depositions on the microstructure and hardness is discussed and a distinctive grain size distribution along the build direction was shown to be a fundamental characteristic.