Never use hardware wallets (such as Ledger) with Polkadot.js. If you have already moved your funds there, they're currently stuck and you can't move them until the SORA Network is supported by the respective hardware wallet. Currently, there is no ETA for integration, but developers and community members are making progress.