Qwen2.5-VL 零样本目标检测实战：提示词定位与 自定义数据集测试

Qwen2.5-VL 零样本目标检测实战提示词定位与 自定义数据集测试这篇教程是我根据 Qwen2.5-VL 多模态定位能力和零样本目标检测复现过程整理出来的。重点演示如何用自然语言 prompt 让模型输出坐标再用 supervision 转换为检测框可视化。Qwen2.5-VL 可以根据文本描述定位图像中的目标。本文保留原始英文 prompt、模型 ID 和推理参数避免改变模型输出。本文会重点跑通以下流程加载 Qwen2.5-VL 并构造视觉问答输入用英文 prompt 控制检测目标和空间约束将 Qwen 输出解析为supervision.Detections在示例图片和 自定义数据集上验证零样本检测效果如果你正在系统学习计算机视觉、多模态模型或 项目复现建议收藏本文配套 notebook、示例图片和运行环境说明后续会继续整理。如果环境配置卡住可以在评论区说明具体报错。 文章目录Qwen2.5-VL 零样本目标检测实战提示词定位与 自定义数据集测试⚙️ 环境准备 加载 Qwen2.5-VL 模型️ 推理与可视化工具函数 示例 1检测 each chair 示例 2检测坐着人的椅子 示例 3多类别检测 示例 4检测玻璃杯➡️ 示例 5检测最右侧玻璃杯 示例 6检测吸管 示例 7检测 pepper 和 salt 在自定义数据集上测试 小结 同系列教程汇总⚙️ 环境准备配置 HuggingFace Token 和数据集后台 API Key检查 GPU安装 maestro 与 supervision。importosfromgoogle.colabimportuserdata os.environ[HF_TOKEN]userdata.get(HF_TOKEN)!nvidia-smi!pip install-qmaestro[qwen_2_5_vl]1.1.0rc2!pip install-q supervision0.26.0# 后台可私信获得下载数据集上传到当前目录或替换为对应资源链接。 加载 Qwen2.5-VL 模型使用 maestro 封装加载 Qwen2.5-VL-7B-Instruct。frommaestro.trainer.models.qwen_2_5_vl.checkpointsimportload_model,OptimizationStrategy MODEL_ID_OR_PATHQwen/Qwen2.5-VL-7B-InstructMIN_PIXELS512*28*28MAX_PIXELS2048*28*28processor,modelload_model(model_id_or_pathMODEL_ID_OR_PATH,optimization_strategyOptimizationStrategy.NONE,min_pixelsMIN_PIXELS,max_pixelsMAX_PIXELS)️ 推理与可视化工具函数定义推理函数和图片标注函数后续所有示例复用这套流程。fromPILimportImagefromtypingimportOptional,Tuple,Unionfrommaestro.trainer.models.qwen_2_5_vl.inferenceimportpredict_with_inputsfrommaestro.trainer.models.qwen_2_5_vl.loadersimportformat_conversationfrommaestro.trainer.common.utils.deviceimportparse_device_specfromqwen_vl_utilsimportprocess_vision_infodefrun_qwen_2_5_vl_inference(model,processor,image:Image.Image,prompt:str,system_message:Optional[str]None,device:strauto,max_new_tokens:int1024,)-Tuple[str,Tuple[int,int]]:deviceparse_device_spec(device)conversationformat_conversation(imageimage,prefixprompt,system_messagesystem_message)textprocessor.apply_chat_template(conversation,tokenizeFalse,add_generation_promptTrue)image_inputs,_process_vision_info(conversation)inputsprocessor(texttext,imagesimage_inputs,return_tensorspt,)input_hinputs[image_grid_thw][0][1]*14input_winputs[image_grid_thw][0][2]*14responsepredict_with_inputs(**inputs,modelmodel,processorprocessor,devicedevice,max_new_tokensmax_new_tokens)[0]returnresponse,(int(input_w),int(input_h))importsupervisionassvfromPILimportImagedefannotate_image(image:Image,detections:sv.Detections)-Image:text_scalesv.calculate_optimal_text_scale(resolution_whimage.size)thicknesssv.calculate_optimal_line_thickness(resolution_whimage.size)ifdetections.maskisnotNone:mask_annotatorsv.MaskAnnotator(color_lookupsv.ColorLookup.INDEX)imagemask_annotator.annotate(image,detections)else:box_annotatorsv.BoxAnnotator(color_lookupsv.ColorLookup.INDEX,thicknessthickness)imagebox_annotator.annotate(image,detections)label_annotatorsv.LabelAnnotator(color_lookupsv.ColorLookup.INDEX,text_colorsv.Color.BLACK,text_scaletext_scale,text_thicknessthickness-1,smart_positionTrue)imagelabel_annotator.annotate(image,detections)returnimage 示例 1检测 each chair用 prompt 定位图中每一把椅子。IMAGE_PATH/content/dog-3.jpegSYSTEM_MESSAGENonePROMPTOutline the position of each chair and output all the coordinates in JSON format.imageImage.open(IMAGE_PATH)resolution_whimage.size response,input_whrun_qwen_2_5_vl_inference(modelmodel,processorprocessor,imageimage,promptPROMPT,system_messageSYSTEM_MESSAGE)print(response)importsupervisionassv detectionssv.Detections.from_vlm(vlmsv.VLM.QWEN_2_5_VL,resultresponse,input_whinput_wh,resolution_whresolution_wh)imageannotate_image(imageimage,detectionsdetections)image.thumbnail((800,800))image 示例 2检测坐着人的椅子通过更具体的描述约束目标范围。IMAGE_PATH/content/dog-3.jpegSYSTEM_MESSAGENonePROMPTOutline the position of chair with man sitting on it and output all the coordinates in JSON format.imageImage.open(IMAGE_PATH)resolution_whimage.size response,input_whrun_qwen_2_5_vl_inference(modelmodel,processorprocessor,imageimage,promptPROMPT,system_messageSYSTEM_MESSAGE)print(response)importsupervisionassv detectionssv.Detections.from_vlm(vlmsv.VLM.QWEN_2_5_VL,resultresponse,input_whinput_wh,resolution_whresolution_wh)imageannotate_image(imageimage,detectionsdetections)image.thumbnail((800,800))image 示例 3多类别检测一次给出多个类别观察模型对复杂 prompt 的处理。IMAGE_PATH/content/dog-3.jpegSYSTEM_MESSAGENonePROMPTOutline the position of chair, dog, table, shoe, light bulb, coffee, hat, glasses, car, tail, umbrella and output all the coordinates in JSON format.imageImage.open(IMAGE_PATH)resolution_whimage.size response,input_whrun_qwen_2_5_vl_inference(modelmodel,processorprocessor,imageimage,promptPROMPT,system_messageSYSTEM_MESSAGE)print(response)IMAGE_PATH/content/dog-3.jpegSYSTEM_MESSAGENonePROMPTOutline the position of chair, dog, table, shoe, coffee, hat, glasses, car, tail, umbrella and output all the coordinates in JSON format.imageImage.open(IMAGE_PATH)resolution_whimage.size response,input_whrun_qwen_2_5_vl_inference(modelmodel,processorprocessor,imageimage,promptPROMPT,system_messageSYSTEM_MESSAGE)print(response)importsupervisionassv detectionssv.Detections.from_vlm(vlmsv.VLM.QWEN_2_5_VL,resultresponse,input_whinput_wh,resolution_whresolution_wh)imageannotate_image(imageimage,detectionsdetections)image.thumbnail((800,800))image 示例 4检测玻璃杯在另一张图片上测试单类别定位。IMAGE_PATH/content/dog-2.jpegSYSTEM_MESSAGENonePROMPTOutline the position of glass and output all the coordinates in JSON format.imageImage.open(IMAGE_PATH)resolution_whimage.size response,input_whrun_qwen_2_5_vl_inference(modelmodel,processorprocessor,imageimage,promptPROMPT,system_messageSYSTEM_MESSAGE)print(response)importsupervisionassv detectionssv.Detections.from_vlm(vlmsv.VLM.QWEN_2_5_VL,resultresponse,input_whinput_wh,resolution_whresolution_wh)imageannotate_image(imageimage,detectionsdetections)image.thumbnail((800,800))image➡️ 示例 5检测最右侧玻璃杯用空间关系约束目标。IMAGE_PATH/content/dog-2.jpegSYSTEM_MESSAGENonePROMPTOutline the position of glass most to the right and output all the coordinates in JSON format.imageImage.open(IMAGE_PATH)resolution_whimage.size response,input_whrun_qwen_2_5_vl_inference(modelmodel,processorprocessor,imageimage,promptPROMPT,system_messageSYSTEM_MESSAGE)print(response)importsupervisionassv detectionssv.Detections.from_vlm(vlmsv.VLM.QWEN_2_5_VL,resultresponse,input_whinput_wh,resolution_whresolution_wh)imageannotate_image(imageimage,detectionsdetections)image.thumbnail((800,800))image 示例 6检测吸管继续测试细长目标定位。IMAGE_PATH/content/dog-2.jpegSYSTEM_MESSAGENonePROMPTOutline the position of straw and output all the coordinates in JSON format.imageImage.open(IMAGE_PATH)resolution_whimage.size response,input_whrun_qwen_2_5_vl_inference(modelmodel,processorprocessor,imageimage,promptPROMPT,system_messageSYSTEM_MESSAGE)print(response)importsupervisionassv detectionssv.Detections.from_vlm(vlmsv.VLM.QWEN_2_5_VL,resultresponse,input_whinput_wh,resolution_whresolution_wh)imageannotate_image(imageimage,detectionsdetections)image.thumbnail((800,800))image 示例 7检测 pepper 和 salt测试两个相近小物体的定位。IMAGE_PATH/content/dog-2.jpegSYSTEM_MESSAGENonePROMPTOutline the position of pepper, salt and output all the coordinates in JSON format.imageImage.open(IMAGE_PATH)resolution_whimage.size response,input_whrun_qwen_2_5_vl_inference(modelmodel,processorprocessor,imageimage,promptPROMPT,system_messageSYSTEM_MESSAGE)print(response)importsupervisionassv detectionssv.Detections.from_vlm(vlmsv.VLM.QWEN_2_5_VL,resultresponse,input_whinput_wh,resolution_whresolution_wh)imageannotate_image(imageimage,detectionsdetections)image.thumbnail((800,800))image 在自定义数据集上测试下载安全背心数据集并对测试集图片运行零样本检测。# 如需额外的数据集下载依赖请按数据集后台说明安装。fromtypesimportSimpleNamespace# 从数据集后台下载并解压数据集后修改 DATASET_DIR 指向数据集目录。DATASET_DIR/content/dataset# 修改为数据集后台导出的数据集目录datasetSimpleNamespace(locationDATASET_DIR)importsupervisionassv dssv.DetectionDataset.from_coco(images_directory_pathf{dataset.location}/test,annotations_pathf{dataset.location}/test/_annotations.coco.json,)image_path,_,annotationsds[0]SYSTEM_MESSAGENonePROMPTOutline the position of helmet and output all the coordinates in JSON format.imageImage.open(image_path)resolution_whimage.size response,input_whrun_qwen_2_5_vl_inference(modelmodel,processorprocessor,imageimage,promptPROMPT,system_messageSYSTEM_MESSAGE)print(response)importsupervisionassv detectionssv.Detections.from_vlm(vlmsv.VLM.QWEN_2_5_VL,resultresponse,input_whinput_wh,resolution_whresolution_wh)imageannotate_image(imageimage,detectionsdetections)image.thumbnail((800,800))image 小结这篇教程完整整理了Zero-Shot Object Detection with Qwen2.5-VL的核心复现流程。实际复现时建议先确认 API Key、GPU、依赖版本和数据集路径再逐段运行 notebook。加载 Qwen2.5-VL 并构造视觉问答输入用英文 prompt 控制检测目标和空间约束将 Qwen 输出解析为supervision.Detections在示例图片和 自定义数据集上验证零样本检测效果后续我会继续按源项目顺序整理 项目教程 中的目标检测、实例分割、OCR、多目标跟踪和视觉大模型教程。 同系列教程汇总Google Gemini 3.5 Flash 零样本目标检测教程从提示词到可视化结果GLM-OCR 文档识别实战教程从验证码、公式到车牌 OCRRF-DETR ByteTrack 多目标跟踪实战教程从命令行到 Python 视频轨迹可视化SAM 3 图像分割实战教程文本、框和点提示的多种分割方式SAM 3 视频分割实战教程用文本提示分割并跟踪视频中的目标