The WRONG way to significantly improve your models:
• Different architecture
• Hyperparameter tuning
• Bigger ensemble
The RIGHT way:
• Thorough EDA
• Meticulous error analysis
• Robust model evaluation
Spend your time on the 100%+ gains, not the 1% gains.