Re: [PATCH] RFC: fmc: Try to convert to GPIO descriptors

[Federico Vaga <federico.vaga@xxxxxxx>]

On Thursday, June 6, 2019 3:58:35 PM CEST Enrico Weigelt, metux IT consult 
wrote:
> On 05.06.19 09:55, Federico Vaga wrote:
> > Hello Enrico,
> > 
> > from your questions is clear that you do not know the fmc-bus and neither
> > the VITA-57.1 (FMC) standard or the IPMI FRU specification.
> 
> Ideed, I never came anywhere near it. (guess, that applies to most
> people here). So, forgive me for asking some stupid questions.
> 
> > Some of your questions
> > are answered by those documents (VITA-57.1, Platform Management FRU
> > Information Storage Definition V1.0, fmc-bus documentation in
> > Documentation/ fmc);
> 
> thanks, I'm just reading the kernel docs (unfortunately lacking the time
> to work through the whole specs).
> 
> Let me some more stupid questions:
> 
> * am I'm correctly assuming that there can be different FMCs, which in
>   turn can be deployed with different application specific bitfiles ?

yes

> * could the SDB data be dynamically changed at runtime ?
>   does it make sense to expose it as an actual fs to userland ?
> * is not having the SDB data a common usecase ?

SDB is something that has been attached to FMC but actually is totally 
unrelated. There were plans about it in the past but they have been abandon.

> * GPIO: are these actually in the carrier itself (not on the cards
>   plugged into it) ? Who's gonna consume them ? (also userland ?)
>   Can we implement this as generic gpio drivers and use
>   gpiod_lookup_table ?

There are no relations between GPIO and FMC. GPIO usage is design specific, 
and in particular specific to the carrier not the modules.


> * chardev: is some kinda /dev/mem for the fmc devices ?
>   is it used by applications or just development ?

probably development. At least at CERN nobody uses it; it has been used to 
hack around at the very beginning

> * eeprom: should we introduce an actual eeprom device class ?
>   several years ago, somebody was working on that, but don't know
>   what happended it it:
>   https://lore.kernel.org/patchwork/patch/436179/

I do not think is necessary. The kernel has support for I2C, it has support to 
the FMC standard EEPROM. They are easy to configure

> By the way: looks like the git repos aren't publiclly accessible:
> (gitlab web frontend seems to work)
>
> >> nekrad@orion:~/src/fmc$ git clone -vvvv
> 
> git://ohwr.org/hdl-core-lib/fpga-config-space.git
>
> >> Cloning into 'fpga-config-space'...
> >> fatal: unable to connect to ohwr.org:
> >> ohwr.org[0: 188.185.83.37]: errno=Connection timed out
> >> 
> >> nekrad@orion:~/src/fmc$ git clone
> 
> https://ohwr.org/project/fpga-config-space.git
> 
> >> Cloning into 'fpga-config-space'...
> >> fatal: unable to access
> 
> 'https://ohwr.org/project/fpga-config-space.git/': gnutls_handshake()
> failed: Handshake failed

I can report the error, but as you can see that project is abandon. 

> >> So, this is some special backplane bus ?
> > 
> > Despite the fact that FMC on Linux is a Linux bus (a mechanism to match
> > devices and driver with some logic), it is not a real bus. Form software
> > prospective it is an I2C EEPROM that identifies an FMC module. A part from
> > this connection with the host, the FMC module is not accessible. All
> > accesses pass through an FPGA which interface is not in a one-to-one
> > relation with any FMC module.
> > 
> > VITA-57.1 standardize the connection between an FPGA and an FMC module. In
> > addition it offers this I2C interface to identify the connected FMC
> > module(s).
> I guess the connection to the host is very FMC specific. Which
> IO operations can we do here ? Just register read/write ?

It is not FMC specific. FMC is just a connector that you can use to extend a 
PCI/VME/USB/ISA/SPI/... card

> Might that be a case for regmap ?
> 
> I'm imagining giving struct fmc_operations a ref (or get/put) to a
> regmap instance instead of readl()/writel() operations. That way,
> subdevice drivers (for some generic logic inside the fpga, eg. a uart,
> gpio, etc) don't need to be fmc-aware at all - they'd just be
> initialized by some bitfile-specific parent driver.
> 
> By the way: is there already name for bitfile specific devices ?
> (maybe call it "fmc payload" ?)
> 
> >> Is that information ("card X is in slot Y", correct ?) important for the
> >> drivers or even userland ?
> > 
> > This sort of information is always important when you what to be able to
> > distinguish cards when you look at then with your eyes, or when those FMC
> > modules are used to create a network with external equipment
> 
> Aha, so it's something that an operator needs to know, like "this is
> the first uart" or "that's the 3rd eth port" ?

Yes
 
> >> Are there some known cases where a different eeprom is used ?
> > 
> > For example, all open-hardware designs that my group at CERN is producing,
> > but there are more out there which are "not standard". This happened
> > because, as I wrote and VITA admitted, the standard is open to
> > misinterpretation; VITA promised to fix it on the next revision.
> 
> hmm, is there any way for probing the eeprom type or is that something
> the operator needs to know/configure ?

In principle there is only one correct EEPROM: 24C02. Any other eeprom is non-
standard, so the operator must know what to do in that specific case.

> Not sure whether I really understood the whole, so I'll try to
> summarize - correct me if I'm wrong:
> 
> * each FMC carries an i2c eeprom with the SDB data, and potentially
>   extra payload specific data (eg. sdbfs)

No.
FMc Modules (plugged on FMC carriers) carry an I2C EEPROM with the "Platform
Management FRU Information Storage Definition V1.0."

> * there may be different eeprom types (speaking different protocols)

According to the standard no. But since the standard is unclear, people 
misunderstood this point, so it is possible to have different eeproms

> * this i2c eeprom is connected to an i2c controller on the carrier

Yes

> * cpu can talk to the carrier in order to do i2c transactions
>   (thus, from cpu pov, the carrier implements an an i2c controller
>   that's connected to that eeprom)

yes

> * the i2c controller type (more precisely: the cpu-visible protocol)
>   is known by carrier type

yes

> Are this standard i2c controllers that are already supported by the
> i2c subsystem ?

There is not a standard I2C controller. It is a carrier design choice

> >>  > As stated before, there is no such thing as an "FMC module driver",
> >>  > nor a "device driver for the FPGA code".
> >> 
> >> Oh, seems I had a wrong idea of that. Do the individual module types
> >> differ - from cpu pov - in anyway, despite the bitfile specific
> >> interfaces ?
> > 
> > I am not sure to understand your question
> 
> My question is whether the FMCs, besides the current bitfile in FPGA,
> differ from host point of view. IOW: does the host need to care about
> the actual FMC type, before we have an actual bitfile loaded ?
> (maybe fpga specific upload protocols or check whether bitfile fits
> that FMC ?)

It depends on the context. If your FPGA is empty and you want to load a new 
configuration then you should check the type of FMC Module plugged in order to 
avoid to "burn" it (really rare, but it could happen)

> >> Who shall be responsible for instantiating these platform devices ?
> >> Maybe oftree overlays ?
> > 
> > Could be DeviceTree, ACPI descriptors, or another driver. There are plenty
> > of possibilities. I have noticed that DeviceTree is extensively used in
> > some context.
> 
> So, this part isn't defined yet ?

It does not need to be defined. It is out of scope. How FPGA devices are 
declare is a general problem, not and FMC problem

> How is that currrently handled ?

The way fmc-bus is designed, does not need to discover FPGA devices.

> Could you put a dtb directly in the FPGA, at some defined address ?

In general yes. Why not?
 
> Some quick idea floating around in my head: we could place a dtb
> right next to the bitfile (one dtb per bitfile), and when fpga payload
> is up and running, start dt-based probing of platform devices.

Yes it is a possibility but 

> hmm, there're several things, I'd like to put on my fun-project-list.

Well, I am suggesting to erase fmc-bus from the Linux kernel and to provide a 
device_class for a standardized way to access that I2C EEPROM. I already have 
the implementation; perhaps it needs some clean-up and for sure to be ported 
to 5.x (I will do this job only if there is interest in removing/replacing the 
current fmc subsystem).

I can tell you that it is not that funny :D
it is only 484 line of code, most of them to handle the special case where the 
EEPROM is not-standard.

> Do you happen to have some remote-accesible testbed for playground ?

Of course not :D
CERN infrastructure is not freely accessible

> 
> --mtx