Toybrick

标题: toybrick rk3399pro PCIe交流 [打印本页]
作者: mahongwei    时间: 2021-3-10 14:38
标题: toybrick rk3399pro PCIe交流
目前使用了PCIe接口做rc 连接了nvme、5g模块、pcie转usb模块等等，都可以完美运行，想玩玩它的ep功能，但是发现：
1.现在的内核是4.4,对pcie ep功能不支持不够好，最少要4.20之后才支持，linux5.4里有些rk3399板子的dts,简单移植了下，起不来
2.要么移植5.4的pci子系统到4.4上
有没有有过经验的人给下帮助，简单的意见也可以

移植5.4内核后启动不了的log：
[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
[    0.000000] Linux version 5.4.0 (mhw@mhw-VirtualBox) (gcc version 6.3.1 20170404 (Linaro GCC 6.3-2017.05)) #2 SMP Tue Mar 9 15:47:03 CST 2021
[    0.000000] Machine model: rockchip,rk3399pro-toybrick-prod-linux
[    0.000000] earlycon: uart8250 at MMIO32 0x00000000ff1a0000 (options '')
[    0.000000] printk: bootconsole [uart8250] enabled
[    0.000000] OF: fdt: Reserved memory: failed to reserve memory for node 'drm-logo@00000000': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt: Reserved memory: failed to reserve memory for node 'stb-devinfo@00000000': base 0x0000000000000000, size 0 MiB
[    0.000000] cma: Reserved 256 MiB at 0x000000006ac00000
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.0 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: Trusted OS migration not required
[    0.000000] psci: SMC Calling Convention v1.0
[    0.000000] percpu: Embedded 30 pages/cpu s81960 r8192 d32728 u122880
[    0.000000] Detected VIPT I-cache on CPU0
[    0.000000] CPU features: detected: ARM erratum 845719
[    0.000000] CPU features: detected: GIC system register CPU interface
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 507912
[    0.000000] Kernel command line: earlycon=uart8250,mmio32,0xff1a0000 root=PARTUUID=614e0000-0000-4b53-8000-1d28000054a9 rw rootwait rootfstype=ext4 swiotlb=1 coherent_pool=1m
[    0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    0.000000] mem auto-init: stackff, heap allocff, heap freeff
[    0.000000] Memory: 1729284K/2064384K available (10366K kernel code, 1504K rwdata, 3776K rodata, 1216K init, 506K bss, 72956K reserved, 262144K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=6, Nodes=1
[    0.000000] ftrace: allocating 41967 entries in 164 pages
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu:         RCU event tracing is enabled.
[    0.000000] rcu:         RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=6.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=6
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[    0.000000] GICv3: 256 SPIs implemented
[    0.000000] GICv3: 0 Extended SPIs implemented
[    0.000000] GICv3: Distributor has no Range Selector support
[    0.000000] GICv3: 16 PPIs implemented
[    0.000000] GICv3: no VLPI support, no direct LPI support
[    0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000fef00000
[    0.000000] ITS [mem 0xfee20000-0xfee3ffff]
[    0.000000] ITS@0x00000000fee20000: allocated 65536 Devices @a300000 (flat, esz 8, psz 64K, shr 0)
[    0.000000] ITS: using cache flushing for cmd queue
[    0.000000] GICv3: using LPI property table @0x000000000a2d0000
[    0.000000] GIC: using cache flushing for LPI property table
[    0.000000] GICv3: CPU0: using allocated LPI pending table @0x000000000a2e0000
[    0.000000] GICv3: GIC: PPI partition interrupt-partition-0[0] { /cpus/cpu@0[0] /cpus/cpu@1[1] /cpus/cpu@2[2] /cpus/cpu@3[3] }
[    0.000000] GICv3: GIC: PPI partition interrupt-partition-1[1] { /cpus/cpu@100[4] /cpus/cpu@101[5] }
[    0.000000] random: get_random_bytes called from start_kernel+0x324/0x4c0 with crng_init=0
[    0.000000] arch_timer: cp15 timer(s) running at 24.00MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x588fe9dc0, max_idle_ns: 440795202592 ns
[    0.000006] sched_clock: 56 bits at 24MHz, resolution 41ns, wraps every 4398046511097ns
[    0.001914] Console: colour dummy device 80x25
[    0.002340] printk: console [tty0] enabled
[    0.002736] printk: bootconsole [uart8250] disabled





作者: mahongwei    时间: 2021-3-18 10:25
目前我选择第二种方法：移植5.4的pci子系统到4.4上,目前controler和endpoint 部分已经移植进去，并且5.4的host 部分已经可以使用。
  1. mhw@mhw-VirtualBox:~/rk3399/kernel/drivers/pci$ tree endpoint/
  2. endpoint/
  3. ├── built-in.o
  4. ├── functions
  5. │   ├── built-in.o
  6. │   ├── Kconfig
  7. │   ├── Makefile
  8. │   ├── modules.builtin
  9. │   ├── modules.order
  10. │   ├── pci-epf-test.c
  11. │   └── pci-epf-test.o
  12. ├── Kconfig
  13. ├── Makefile
  14. ├── modules.builtin
  15. ├── modules.order
  16. ├── pci-epc-core.c
  17. ├── pci-epc-core.o
  18. ├── pci-ep-cfs.c
  19. ├── pci-ep-cfs.o
  20. ├── pci-epc-mem.c
  21. ├── pci-epc-mem.o
  22. ├── pci-epf-core.c
  23. └── pci-epf-core.o
  24. mhw@mhw-VirtualBox:~/rk3399/kernel/drivers/pci$ tree controller/
  25. controller/
  26. ├── built-in.o
  27. ├── Kconfig
  28. ├── Makefile
  29. ├── modules.builtin
  30. ├── modules.order
  31. ├── pcie-rockchip.c
  32. ├── pcie-rockchip-ep.c
  33. ├── pcie-rockchip-ep.o
  34. ├── pcie-rockchip.h
  35. ├── pcie-rockchip-host.c
  36. ├── pcie-rockchip-host.o
  37. ├── pcie-rockchip.o
  38. ├── pci-thunder-ecam.c
  39. ├── pci-thunder-pem.c
  40. └── pci-xgene.c
复制代码

0 directories, 15 files
开机成功，然后依靠configfs进行了设置：

  1. root@debian10:/home/toybrick# cd /sys/kernel/config/pci_ep/
  2. root@debian10:/sys/kernel/config/pci_ep# mkdir functions/pci_epf_test/func1
  3. root@debian10:/sys/kernel/config/pci_ep# ls functions/pci_epf_test/func1
  4. baseclass_code         interrupt_pin          progif_code         subsys_id
  5. cache_line_size  msi_interrupts   revid                 subsys_vendor_id
  6. deviceid         msix_interrupts  subclass_code  vendorid
  7. root@debian10:/sys/kernel/config/pci_ep# echo 0x104c > functions/pci_epf_test/func1/vendorid
  8. root@debian10:/sys/kernel/config/pci_ep# echo 0xb500 > functions/pci_epf_test/func1/deviceid
  9. root@debian10:/sys/kernel/config/pci_ep# ln -s functions/pci_epf_test/func1 controllers/fd000000.pcie-ep/
  10. [   29.881158] [pci_epc_epf_link:97]mhwpcie start
  11. [   29.884855] [rockchip_pcie_ep_set_bar:173]mhwpcie start!
  12. [   29.885348] [rockchip_pcie_ep_set_bar:229]mhwpcie end!
  13. [   29.885911] [rockchip_pcie_ep_set_bar:173]mhwpcie start!
  14. [   29.886399] [rockchip_pcie_ep_set_bar:229]mhwpcie end!
  15. [   29.886911] [rockchip_pcie_ep_set_bar:173]mhwpcie start!
  16. [   29.887398] [rockchip_pcie_ep_set_bar:229]mhwpcie end!
  17. [   29.887903] [rockchip_pcie_ep_set_bar:173]mhwpcie start!
  18. [   29.888389] [rockchip_pcie_ep_set_bar:229]mhwpcie end!
  19. [   29.888931] [rockchip_pcie_ep_set_bar:173]mhwpcie start!
  20. [   29.889418] [rockchip_pcie_ep_set_bar:229]mhwpcie end!
  21. [   29.889921] [rockchip_pcie_ep_set_bar:173]mhwpcie start!
  22. [   29.890407] [rockchip_pcie_ep_set_bar:229]mhwpcie end!
  23. [   29.890981] [pci_epc_epf_link:115]mhwpcie end
  24. root@debian10:/sys/kernel/config/pci_ep# echo 1 > controllers/fd000000.pcie-ep/start
  25. [   29.893927] [rockchip_pcie_ep_start:502]mhwpcie start!
  26. [   29.894399] [rockchip_pcie_ep_start:508]mhwpcie end!
复制代码

感觉上ep模式已经和linux5.4 mainline上的设置一致了，并且没有报错。
作者: mahongwei    时间: 2021-3-18 10:36
然后我买了个pcie延长线R11NS，并且把rx和tx 反接了
https://item.taobao.com/item.htm ... amp;id=608821495198
，插入一个其他带pcie2.0 host的开发版,这里成为host，ep(rk3399pro)先启动，host启动时，在读取pcie vid时报错：
  1. [    1.773968] nwl-pcie fd0e0000.pcie: Link is UP
  2. [    1.777832] OF: PCI: host bridge /amba/pcie@fd0e0000 ranges:
  3. [    1.783430] OF: PCI:   MEM 0xe0000000..0xefffffff -> 0xe0000000
  4. [    1.789303] OF: PCI:   MEM 0x600000000..0x7ffffffff -> 0x600000000
  5. [    1.795556] nwl-pcie fd0e0000.pcie: PCI host bridge to bus 0000:00
  6. [    1.801582] pci_bus 0000:00: root bus resource [bus 00-ff]
  7. [    1.807028] pci_bus 0000:00: root bus resource [mem 0xe0000000-0xefffffff]
  8. [    1.813861] pci_bus 0000:00: root bus resource [mem 0x600000000-0x7ffffffff pref]
  9. [    1.821331] pci 0000:00:00.0: ignoring class 0x068000 (doesn't match header type 01)
  10. [    3.987047] Synchronous External Abort: synchronous external abort (0x96000210) at 0xffffff800b100000
  11. [    3.990606] Internal error: : 96000210 [#1] SMP
  12. [    3.995103] Modules linked in:
  13. [    3.998133] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.14.0-xilinx-v2018.3 #10
  14. [    4.005396] Hardware name: xlnx,zynqmp (DT)
  15. [    4.009549] task: ffffffc87d858d00 task.stack: ffffff8008038000
  16. [    4.015438] PC is at pci_generic_config_read+0x34/0xa8
  17. [    4.020535] LR is at pci_generic_config_read+0x20/0xa8
  18. [    4.025638] pc : [<ffffff800847f824>] lr : [<ffffff800847f810>] pstate: 200000c5
  19. [    4.032989] sp : ffffff800803b990
  20. [    4.036276] x29: ffffff800803b990 x28: ffffffc87ccaa000
  21. [    4.041553] x27: 0000000000000001 x26: 0000000000000000
  22. [    4.046829] x25: 000000000000000c x24: 0000000000000040
  23. [    4.052106] x23: 0000000000000000 x22: ffffff800803ba8c
  24. [    4.057383] x21: ffffff8008f02bc0 x20: ffffff800803b9fc
  25. [    4.062659] x19: 0000000000000004 x18: 0000000000000000
  26. [    4.067935] x17: 0000000000000001 x16: 0000000000000019
  27. [    4.073212] x15: ffffffffffffffff x14: ffffffc87da8591c
  28. [    4.078489] x13: ffffffc87da85198 x12: 0000000000000000
  29. [    4.083765] x11: 0101010101010101 x10: 7f7f7f7f7f7f7f7f
  30. [    4.089042] x9 : ffffff8008e0a000 x8 : ffffffc87d371e80
  31. [    4.094318] x7 : 0000000000000000 x6 : 0000000000000001
  32. [    4.099595] x5 : 0000000000000000 x4 : ffffffc87d254000
  33. [    4.104871] x3 : ffffffc87ccaab98 x2 : 0000000000000000
  34. [    4.110148] x1 : ffffff800b000000 x0 : ffffff800b100000
  35. [    4.115426] Process swapper/0 (pid: 1, stack limit = 0xffffff8008038000)
  36. [    4.122085] Call trace:
  37. [    4.124509] Exception stack(0xffffff800803b850 to 0xffffff800803b990)
  38. [    4.130910] b840:                                   ffffff800b100000 ffffff800b000000
  39. [    4.138696] b860: 0000000000000000 ffffffc87ccaab98 ffffffc87d254000 0000000000000000
  40. [    4.146481] b880: 0000000000000001 0000000000000000 ffffffc87d371e80 ffffff8008e0a000
  41. [    4.154266] b8a0: 7f7f7f7f7f7f7f7f 0101010101010101 0000000000000000 ffffffc87da85198
  42. [    4.162051] b8c0: ffffffc87da8591c ffffffffffffffff 0000000000000019 0000000000000001
  43. [    4.169836] b8e0: 0000000000000000 0000000000000004 ffffff800803b9fc ffffff8008f02bc0
  44. [    4.177621] b900: ffffff800803ba8c 0000000000000000 0000000000000040 000000000000000c
  45. [    4.185406] b920: 0000000000000000 0000000000000001 ffffffc87ccaa000 ffffff800803b990
  46. [    4.193191] b940: ffffff800847f810 ffffff800803b990 ffffff800847f824 00000000200000c5
  47. [    4.200976] b960: 0000000000000759 0000000000000000 ffffffffffffffff 0000000000000000
  48. [    4.208760] b980: ffffff800803b990 ffffff800847f824
  49. [    4.213607] [<ffffff800847f824>] pci_generic_config_read+0x34/0xa8
  50. [    4.219747] [<ffffff800847f720>] pci_bus_read_config_dword+0x68/0xb8
  51. [    4.226064] [<ffffff80084820b4>] pci_bus_read_dev_vendor_id+0x2c/0x1a0
  52. [    4.232550] [<ffffff80084838a4>] pci_scan_single_device+0x54/0xe0
  53. [    4.238605] [<ffffff800848395c>] pci_scan_slot+0x2c/0xf0
  54. [    4.243881] [<ffffff80084849c8>] pci_scan_child_bus+0x30/0x118
  55. [    4.249677] [<ffffff800848465c>] pci_scan_bridge+0x1e4/0x520
  56. [    4.255300] [<ffffff8008484a24>] pci_scan_child_bus+0x8c/0x118
  57. [    4.261095] [<ffffff8008484bec>] pci_scan_root_bus_bridge+0x54/0xd8
  58. [    4.267323] [<ffffff800849baac>] nwl_pcie_probe+0x794/0xa18
  59. [    4.272860] [<ffffff8008586368>] platform_drv_probe+0x58/0xb8
  60. [    4.278569] [<ffffff800858474c>] driver_probe_device+0x22c/0x2d8
  61. [    4.284537] [<ffffff80085848b4>] __driver_attach+0xbc/0xc0
  62. [    4.289986] [<ffffff800858288c>] bus_for_each_dev+0x4c/0x98
  63. [    4.295522] [<ffffff8008584048>] driver_attach+0x20/0x28
  64. [    4.300799] [<ffffff8008583b98>] bus_add_driver+0x1b8/0x228
  65. [    4.306334] [<ffffff8008585260>] driver_register+0x60/0xf8
  66. [    4.311784] [<ffffff80085862b8>] __platform_driver_register+0x40/0x48
  67. [    4.318188] [<ffffff8008dbf694>] nwl_pcie_driver_init+0x18/0x20
  68. [    4.324068] [<ffffff8008083980>] do_one_initcall+0x38/0x128
  69. [    4.329605] [<ffffff8008da0cc4>] kernel_init_freeable+0x138/0x1d8
  70. [    4.335659] [<ffffff8008a5ce48>] kernel_init+0x10/0x100
  71. [    4.340848] [<ffffff8008084a90>] ret_from_fork+0x10/0x18
  72. [    4.346126] Code: 7100067f 54000140 71000a7f 54000200 (b9400001)
  73. [    4.352181] ---[ end trace 10ac5e592c4ebb6e ]---
  74. [    4.356774] nwl-pcie fd0e0000.pcie: Slave error
  75. [    4.361290] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
  76. [    4.361290]
  77. [    4.370344] SMP: stopping secondary CPUs
复制代码




欢迎光临 Toybrick (https://t.rock-chips.com/) Powered by Discuz! X3.3