Overfitting vs. underfitting

In the context of statistical modeling and machine learning, what is the difference betweeen overfitting and underfitting? At a high level, what steps can you take to avoid these issues?