Common mineral soil contaminants include arsenic, barium, cadmium, copper, mercury, lead and zinc.
Lead is a particularly dangerous soil component. The following table from the University of Minnesota categorizes typical soil concentration levels and their associated health risks.
Six gardening practices to reduce the lead risk
Locate gardens away from old painted structures and heavily traveled roads
Give planting preferences to fruiting crops (tomatoes, squash, peas, sunflowers, corn, etc.)
Incorporate organic materials such as finished compost, humus, and peat moss
Lime soil as recommended by soil test (pH 6.5 minimizes lead availability)
Discard old and outer leaves before eating leafy vegetables; peel root crops; wash all produce
Keep dust to a minimum by maintaining a mulched and/or moist soil surface
Reference: https://en.wikipedia.org/wiki/Soil_test