You need valid XHTML if you want some features of mootools to work properly. In general, you want valid XHTML with a transitional or strict document type (doctype).
Use one of the following doctypes.
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
or
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
Keep in mind that while invalidation from W3C isn't the end of the world, it certainly doesn't look good.