SwiftSpec: Ultra-Low Latency LLM Decoding by Scaling Asynchronous Speculative Decoding