Restrict the upload of document types without the compliance kit?

There's no built-in configuration option for this, so you need to either use one of the add-on modules (recommended) or develop an event handler for this yourself.

Thank you Jonas

You could probably use the property calculator to generate an error. 
Use the file name property for filtering.