You need to add a .htaccess entry (in the /panel/.htaccess) with redirect permanent to https at the very top.
Code:
RewriteEngine on
# Force SSL - Always first!
RewriteCond %{SERVER_PORT} 80
RewriteRule ^(.*)$ https://panel.domain.com/$1 [R,L]