Tomatoes are vital in global diets for their nutrients like vitamin C and lycopene, but ripe ones spoil easily and unripe ones contain harmful solanine. Accurate ripeness detection is key to reducing waste and ensuring safety, while manual picking lacks efficiency and standards, driving the need for automated inspection. This study proposes a multi-task deep CNN based on YOLOv11, integrating Swin-Transformer into the backbone (inspired by RT-DETR) to enhance global information processing. New modules like ASSFHead and ENLCA are added to boost feature extraction. Experiments show the optimized model improves mAP50 by 1.6%, mAP50-95 by 0.6%, and recall by 2.2%. The achievement reduces harvesting losses, ensures food safety, offers references for other produce, and promotes agricultural intelligent detection technologies. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2026.