Keep on keeping on. It's not as though the territories they've got are primed for major programs of colonial settlement like the Portuguese tried. Though they might do what Franco did in WWII and occupy Tangier so that might make Spanish (North) Morocco kind of viable as a thing to cling into?