Skip to content

Fix Spider main thread any problem with redis connection (when working with redis-based schedulers) leads to unhandled exception and release the main (control) thread#846

Open
Interfer0n wants to merge 4 commits intocode4craft:masterfrom
Interfer0n:master

Conversation

@Interfer0n
Copy link

Exception in thread "Thread-5" redis.clients.jedis.exceptions.JedisConnectionException: Unexpected end of stream.
    at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:199)
    at redis.clients.util.RedisInputStream.readByte(RedisInputStream.java:40)
    at redis.clients.jedis.Protocol.process(Protocol.java:151)
    at redis.clients.jedis.Protocol.read(Protocol.java:215)
    at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:340)
    at redis.clients.jedis.Connection.getStatusCodeReply(Connection.java:239)
    at redis.clients.jedis.BinaryJedis.select(BinaryJedis.java:523)
    at us.codecraft.webmagic.scheduler.RedisPriorityScheduler.poll(RedisPriorityScheduler.java:63)
    at us.codecraft.webmagic.Spider.run(Spider.java:309)
    at java.lang.Thread.run(Thread.java:748)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant