WarmGrayMatteInkMistClaySage
RELIEF · DEBUG

Six figures you open while the bug is still on screen六张 bug 还挂在屏幕上时会打开的图

Not diagrams for learning how something works — diagrams for holding next to a failure. Each one answers a question that comes up while you are already stuck: why nothing printed, who is waiting on whom, what the count was when it died, and which interval the bug actually lives in.不是用来搞懂原理的图 —— 是出事的时候摆在旁边看的图。每一张回答一个「你已经卡住了才会问」的问题:为什么什么都没打印、谁在等谁、死的时候计数是多少、bug 到底活在哪一段区间里。

1 · Why probe never ran1 · probe 为什么没跑

Five gates, and the first three fail silently — no error, no log line, nothing. That is what makes this the question every driver author asks first. The accent marks the first gate that is inside your own code: if the dev_info() at the top of probe does not print, the problem is above that line, and reading your driver harder will not find it.五道关卡,前三道挡住你的时候一声不吭 —— 不报错,不打日志,什么都没有。这就是为什么它是每个写驱动的人第一个要问的问题。强调色标的是第一道进到你自己代码里的关卡:probe 开头那句 dev_info() 没打印,问题就在这条线以上,再怎么细读自己的驱动也找不着。

DEVICE / DRIVER BINDING设备与驱动的绑定the right-hand column is the actual content of this figure — each gate fails with a different symptom右边那一列才是这张图的正文 —— 每道关卡挡住你的样子都不一样
1
of_match_tablethe compatible string in the driver驱动里写的 compatible 串
No node in the device tree matches. Nothing is printed at all — no error, no probe. This is the silent one.设备树里没有节点跟它对上。什么都不打印 —— 不报错,也不 probe。这一档是最安静的
2
status = "okay"the node must be enabled节点必须是使能的
The node exists but is disabled. Same symptom as above: silence. Check this before you touch the driver.节点在,但被禁用了。症状和上面一样:安静。改驱动之前先查这一条。
3
bus match()platform / i2c / spi — each bus matches differentlyplatform / i2c / spi 各有各的匹配法
i2c matches on the address too, not just compatible. A wrong reg here looks exactly like a wrong compatible.i2c 还要匹配地址,不只是 compatible。reg 写错,看起来和 compatible 写错一模一样
4
probe() is calledprobe() 被调用you are finally inside your own code终于进到自己的代码里了
If you got here, dev_info() at the top of probe prints. If it does not print, the problem is above this line, not in your driver. 走到这一步,probe 开头的 dev_info() 就会打印。它没打印,问题在这条线之上,不在你的驱动里。
5
devm_* gets resourcesdevm_* 拿资源clock, regulator, gpio, irq时钟、稳压器、gpio、中断
A missing supplier returns -EPROBE_DEFER. The kernel retries later, so it may work on the second pass and fail on a cold boot.依赖还没就绪会返回 -EPROBE_DEFER。内核过会儿重试,所以它可能第二轮就好了,冷启动却失败

2 · Who holds what2 · 锁的持有关系

A lock you hold is a thing you have, so it is drawn raised. A lock you are waiting for is an empty socket, so it is drawn sunken and ringed. Read the two lanes and the cycle is visible without following a single arrow — that cycle is the deadlock, and there is nothing else to look for.拿在手里的锁是你有的东西,所以画成凸起。在等的锁是个空槽,所以画成凹陷并描圈。两条泳道并排一看,环自己就出来了,不用顺着任何一根箭头走 —— 那个环就是死锁,别的不用再找。

AB-BA DEADLOCKAB-BA 死锁lockdep prints this same picture as text; the shape is what makes it readable in one glancelockdep 打出来的就是这同一张图,只是文字形式。做成形状才是一眼能看完的
thread A · tp_fw_update()线程 A · tp_fw_update()
&tp->lockholds it拿着
i2c bus lockwaiting for B to let go在等 B 松手
A waits on BA 等 BB waits on A · nobody movesB 等 A · 谁也动不了
thread B · tp_irq_thread()线程 B · tp_irq_thread()
i2c bus lockholds it拿着
&tp->lockwaiting for A to let go在等 A 松手

3 · Reference count lifetime3 · 引用计数生命周期

Each row is one get or put and what the count became. While the object is alive the row is raised; once the count hits zero the object is freed and the last row collapses into a groove. A call that lands on a sunken row is reaching into memory that is gone — that is the whole shape of a use-after-free.每一行是一次 get 或 put,以及计数变成了多少。对象活着的时候行是凸起的;计数一归零对象被释放,最后那行塌成一道凹槽。落在凹行上的调用,就是往已经没有了的内存里伸手 —— 释放后使用的全部形状就是这样。

ONE OBJECT, SEVEN EVENTS一个对象,七次事件the last row is the bug: a timer that outlived the object it points into最后一行就是 bug:一个活得比它指向的对象还久的定时器
probe()
get_device() → 1object allocated, count 1对象分配出来,计数 1
open()
get() → 2a user opened it有用户打开了它
irq thread中断线程
get() → 3the irq path took a reference too中断路径也拿了一份
close()
put() → 2
irq thread中断线程
put() → 1
remove()
put() → 0count reaches 0 — the object is freed here计数归零 —— 对象在这里被释放
a stale timer过期的定时器
put() → —use-after-free. The timer was never cancelled, so it still holds a pointer into freed memory释放后使用。 定时器没被取消,它手里还攥着已释放内存的指针

4 · The race window4 · 竞态窗口

Two lanes share one time axis, and between them sits a sunken band — the window. Everything outside it is fine; the bug exists only while both sides have read the old value and neither has written back. Drawing the window as a groove says that without a sentence: the danger is not an instruction, it is an interval.两条泳道共用一根时间轴,中间夹着一条凹带 —— 那就是窗口。窗口外面一切正常;bug 只存在于两边都读到了旧值、而谁都还没写回去的这段时间里。把窗口画成凹槽,不用写一句话就说清了:危险的不是某条指令,是一段区间。

LOST UPDATE ON A REFCOUNT引用计数的丢失更新add a lock and the window closes; add a print and it moves — which is why it reproduces once a week加把锁窗口就关上了;加句打印它就挪位置 —— 所以这种 bug 一周才复现一次
irq thread中断线程
read cnt = 1
 
 
cnt = 1 - 1 = 0
kfree(obj)
 
the window — both have read 1, neither has written yet. Everything breaks only in here.窗口在这 —— 两边都读到了 1,谁都还没写回去。只有在这一段里才会出事
read() syscallread() 系统调用
 
read cnt = 1
cnt = 1 + 1 = 2
 
 
obj->buf

5 · Reading an oops5 · oops 怎么读

The raw dump goes in a sunken panel, because that is what it is: a screen. Beside each line is what it actually tells you. Most of the value in an oops is in the three lines people skip — the fault address is usually not 0, and the offset is the struct field.原始 dump 放进凹陷面板,因为它本来就是一块屏。每一行旁边写它到底在说什么。一份 oops 的价值大半在被跳过的那三行里 —— 出错地址通常不是 0,那个偏移就是结构体里的字段。

NULL POINTER DEREFERENCE空指针解引用the fault address 0x18 is a field offset, not a pointer — pahole turns it into a field name出错地址 0x18 是字段偏移,不是指针 —— 用 pahole 能把它变成字段名
Unable to handle kernel NULL pointer dereference at virtual address 0000000000000018\npc : tp_read_frame+0x48/0x120 [tp_core]\nlr : tp_irq_thread+0x9c/0x1e0 [tp_core]\nx0 : 0000000000000000 x1 : ffffff8041a2c000\nCall trace:\n tp_read_frame+0x48/0x120\n tp_irq_thread+0x9c/0x1e0
at virtual address 0x18Not 0x0 — so it is not a plain NULL pointer. It dereferenced NULL->field and 0x18 is that field’s offset. Run pahole on the struct and 0x18 names the field.不是 0x0 —— 所以不是纯空指针,是 NULL->某字段,0x18 就是那个字段的偏移。对结构体跑一下 pahole,0x18 直接指出是哪个字段
pc : tp_read_frame+0x48Where it died. Feed the function plus 0x48 to addr2line or gdb and you get the source line.死在哪。 把函数名加 0x48 喂给 addr2line 或 gdb,直接出源码行
lr : tp_irq_thread+0x9cWho called it. The link register still holds the return address, so this is the caller even when the stack is too damaged to walk.谁调的。 链接寄存器里还留着返回地址,所以哪怕栈已经踩烂到走不动,这一行仍然有效
x0 : 0000000000000000The first argument was NULL. On arm64 x0 is arg 1, so the caller passed in the null.第一个参数是空。arm64 上 x0 就是第一个参数,说明空是调用方传进来的
[tp_core]A module, so the addresses are relative to where it was loaded. Compare against the exact build you flashed — the same source built twice does not give the same offsets.是模块,所以地址是相对加载位置的。要拿烧进去的那一版比对 —— 同一份源码编两次,偏移不一样

6 · Why it never woke up6 · 它为什么没醒

Same two lanes, same shape, different question. The sleeper is sunken because it is in the queue; the waker is raised because it is running. The gap between them is the missed wakeup: the condition was set and wake_up() fired before the sleeper actually went to sleep, so the wakeup had nobody to deliver to.同样两条泳道,同样的形状,换一个问题。睡的那个凹下去,因为它在队列里;唤的那个凸起,因为它在跑。中间那道缝就是丢失的唤醒:条件置位、wake_up() 触发,都发生在睡的那个真正睡下之前,于是这次唤醒没有人可以送达。

MISSED WAKEUP丢失的唤醒this is why wait_event() re-checks the condition inside the macro — and why your own loop must too这就是 wait_event() 在宏里面要重新检查条件的原因 —— 你自己写循环也必须这么做
the sleeper睡的那个
wait_event_interruptible()sank into the queue沉进队列了
condition == falseit checked before sleeping睡之前检查过一次
the gap缝在这condition set here → wake_up never reaches it条件在这一刻置位 → wake_up 追不上它
the waker唤的那个
condition = truesets it置位
wake_up()fires — but the sleeper had not slept yet触发了 —— 可这会儿睡的那个还没睡下

7 · Where an overflow lands7 · 越界落在哪

The allocator hands out a whole bucket, so a 32-byte object sits in 64 bytes with redzones on both sides, drawn sunken and ringed. Overflow a little and you land in a redzone, which is watched, and you get a report. Overflow past it and you land in the next object, which is watched by nobody — same bug, and the only difference is how far you went.分配器发的是一整格,所以一个 32 字节的对象住在 64 字节里,两边各有一条红区,画成凹陷并描警示色。越界得少,落在红区里,那里有人盯着,你会拿到一份报告。越过红区,落进下一个对象,那里没人盯 —— 同一个 bug,唯一的区别是你走多远。

kmalloc-64 · one 32-byte objectkmalloc-64 · 一个 32 字节的对象this is why an overflow reproduces as someone else’s data going wrong, in a subsystem you never touched这就是为什么越界复现出来,是别人的数据出错,而且在一个你根本没碰过的子系统里
kmalloc-64 #0
REDZONEKASAN fills this and watches itKASAN 往这里填标记然后盯着它
your object32 bytes asked for, 64 handed out要了 32 字节,实际给了 64
REDZONEa 4-byte overflow lands here and is caught越界 4 个字节落在这里,抓得到
kmalloc-64 #2a 40-byte overflow lands here and is silent越界 40 个字节落到这里,一声不吭

8 · Reading a gap in dmesg8 · 读 dmesg 里的空档

Prints are raised, and the stretch with no prints is a sunken band — because in a hang the evidence is not what was printed, it is the interval where nothing was. Timestamps are the only thing that makes that interval visible, which is the argument for CONFIG_PRINTK_TIME being on everywhere.打印是凸的,而中间那段没有打印的画成凹带 —— 因为卡死这类问题,证据不是打印出来的东西,是什么都没有的那段区间。时间戳是唯一能让这段区间显形的东西,这也就是 CONFIG_PRINTK_TIME 到处都该打开的理由。

A 12-SECOND HOLE一个 12 秒的洞the line after the gap names what ended it, and that is usually the fastest route to the cause空档之后那一行说的是「谁结束了它」,而那通常是找到原因最快的一条路
[12.004310]
tp 1-0038: probe start
[12.004982]
tp 1-0038: reset asserted
11.9 seconds with no output at all — this gap IS the finding整整 11.9 秒没有任何输出 —— 这段空档就是结论
[23.918770]
i2c 1: controller timed outthe timeout is what ended the gap是这次超时结束了空档
[23.919104]
tp 1-0038: probe failed -110-110 is ETIMEDOUT-110 就是 ETIMEDOUT