ScrapydWeb 现已支持自定义 Run Spider 页面的 settings & arguments 默认值

1.安装更新:

pip install -U git+https://github.com/my8100/scrapydweb.git 

2.如果之前已在使用 scrapydweb v1.2.0，则在已有的配置文件中添加如下配置选项：

 ############################## Run Spider ##################################### # The default is False, set it to True to automatically # expand the 'settings & arguments' section in the Run Spider page. SCHEDULE_EXPAND_SETTINGS_ARGUMENTS = False # The default is 'Mozilla/5.0', set it a non-empty string to customize the default value of `custom` # in the drop-down list of `USER_AGENT`. SCHEDULE_CUSTOM_USER_AGENT = 'Mozilla/5.0' # The default is None, set it to any value of ['custom', 'Chrome', 'iPhone', 'iPad', 'Android'] # to customize the default value of `USER_AGENT`. SCHEDULE_USER_AGENT = None # The default is None, set it to True or False to customize the default value of `ROBOTSTXT_OBEY`. SCHEDULE_ROBOTSTXT_OBEY = None # The default is None, set it to True or False to customize the default value of `COOKIES_ENABLED`. SCHEDULE_COOKIES_ENABLED = None # The default is None, set it to a non-negative integer to customize the default value of `CONCURRENT_REQUESTS`. SCHEDULE_CONCURRENT_REQUESTS = None # The default is None, set it to a non-negative number to customize the default value of `DOWNLOAD_DELAY`. SCHEDULE_DOWNLOAD_DELAY = None # The default is "-d setting=CLOSESPIDER_TIMEOUT=60\r\n-d setting=CLOSESPIDER_PAGECOUNT=10\r\n-d arg1=val1", # set it to '' or any non-empty string to customize the default value of `additional`. # Use '\r\n' as the line separator. SCHEDULE_ADDITIOnAL= "-d setting=CLOSESPIDER_TIMEOUT=60\r\n-d setting=CLOSESPIDER_PAGECOUNT=10\r\n-d arg1=val1" 

3.GitHub

https://github.com/my8100/scrapydweb