The flow.end field for NEW connections has a value of 0. However it seems that the flowEnd* IPFIX are commonly interpreted as "last seen" timestamps. >From https://www.iana.org/assignments/ipfix/ipfix.xhtml: 153 flowEndMilliseconds The absolute timestamp of the last packet of this Flow. It's not clear whether "last packet" should be read as "final/closing packet", but with this field carrying a value of 0 the nfdump doesn't handle the flowStartMilliseconds value as well. Moreover, NF_NETLINK_CONNTRACK_UPDATE events also set flow.end to timestamp of last packet _seen_, with the connection being still established (UPDATEd connection is on-going per-se, until DESTROY). The actual state of the flow, i.e. it's termination, should be read directly from event type (firewallEvent/natEvent fields), not derived from flowEnd* having non-zero value. Therefore, when flow.end is not set, make it equal to flow.start, as initiating packet and the last one are the same. As this change is only 1 line, I've left the remaining notes and comments here. -- Tomasz Pala <gotar@xxxxxxxxxxxxx>
diff --color '--palette=ad=1;38;5;155:de=1;38;5;205:hd=1;36:ln=35;1;3' -U5 -ru ddd/ulogd2/input/flow/ulogd_inpflow_NFCT.c ./input/flow/ulogd_inpflow_NFCT.c --- ddd/input/flow/ulogd_inpflow_NFCT.c 2023-12-13 12:14:03.137497479 +0100 +++ aaa/input/flow/ulogd_inpflow_NFCT.c 2023-12-13 10:12:46.267523061 +0100 @@ -244,21 +244,20 @@ .flags = ULOGD_RETF_NONE, .name = "orig.raw.pktlen", .ipfix = { .vendor = IPFIX_VENDOR_IETF, .field_id = IPFIX_octetTotalCount, - /* FIXME: this could also be octetDeltaCount */ }, }, { .type = ULOGD_RET_UINT64, .flags = ULOGD_RETF_NONE, .name = "orig.raw.pktcount", .ipfix = { .vendor = IPFIX_VENDOR_IETF, .field_id = IPFIX_packetTotalCount, - /* FIXME: this could also be packetDeltaCount */ + /* FIXME: this could also be egressUnicastPacketTotalCount */ }, }, { .type = ULOGD_RET_IPADDR, .flags = ULOGD_RETF_NONE, @@ -309,21 +308,20 @@ .flags = ULOGD_RETF_NONE, .name = "reply.raw.pktlen", .ipfix = { .vendor = IPFIX_VENDOR_IETF, .field_id = IPFIX_octetTotalCount, - /* FIXME: this could also be octetDeltaCount */ }, }, { .type = ULOGD_RET_UINT64, .flags = ULOGD_RETF_NONE, .name = "reply.raw.pktcount", .ipfix = { .vendor = IPFIX_VENDOR_IETF, .field_id = IPFIX_packetTotalCount, - /* FIXME: this could also be packetDeltaCount */ + /* FIXME: this could also be ingressUnicastPacketTotalCount */ }, }, { .type = ULOGD_RET_UINT8, .flags = ULOGD_RETF_NONE, diff --color '--palette=ad=1;38;5;155:de=1;38;5;205:hd=1;36:ln=35;1;3' -U5 -ru ddd/ulogd2/output/ipfix/ulogd_output_IPFIX.c ./output/ipfix/ulogd_output_IPFIX.c --- ddd/output/ipfix/ulogd_output_IPFIX.c 2023-12-13 12:14:03.137497479 +0100 +++ aaa/output/ipfix/ulogd_output_IPFIX.c 2023-12-13 11:59:52.652179788 +0100 @@ -488,19 +566,21 @@ data->saddr.s_addr = ikey_get_u32(&pi->input.keys[InIpSaddr]); data->daddr.s_addr = ikey_get_u32(&pi->input.keys[InIpDaddr]); data->tsaddr.s_addr = ikey_get_u32(&pi->input.keys[InTIpSaddr]); data->tdaddr.s_addr = ikey_get_u32(&pi->input.keys[InTIpDaddr]); + /* TODO: send full uint64_t */ data->packets = htonl((uint32_t) (ikey_get_u64(&pi->input.keys[InRawInPktCount]) + ikey_get_u64(&pi->input.keys[InRawOutPktCount]))); data->bytes = htonl((uint32_t) (ikey_get_u64(&pi->input.keys[InRawInPktLen]) + ikey_get_u64(&pi->input.keys[InRawOutPktLen]))); start = (uint64_t)ikey_get_u32(&pi->input.keys[InFlowStartSec]) *1000 + ikey_get_u32(&pi->input.keys[InFlowStartUsec])/1000; end = (uint64_t)ikey_get_u32(&pi->input.keys[InFlowEndSec]) *1000 + ikey_get_u32(&pi->input.keys[InFlowEndUsec])/1000; data->start_low = htonl((uint32_t)(start & 0xFFFFFFFFUL)); data->start_high = htonl(start >> 32); + if (!end) end = start; /* end timestamp seems to be commonly read as "last seen" timestamp */ data->end_low = htonl((uint32_t)(end & 0xFFFFFFFFUL)); data->end_high = htonl(end >> 32); if (GET_FLAGS(pi->input.keys, InL4SPort) & ULOGD_RETF_VALID) { data->sport = htons(ikey_get_u16(&pi->input.keys[InL4SPort])); diff --color '--palette=ad=1;38;5;155:de=1;38;5;205:hd=1;36:ln=35;1;3' -U5 -ru ddd/ulogd2/README ./README --- ddd/README 2023-12-13 12:10:38.503561734 +0100 +++ aaa/README 2023-12-10 19:58:55.735776260 +0100 @@ -1,9 +1,9 @@ Userspace logging daemon for netfilter/iptables -Project Homepage: http://www.gnumonks.org/projects/ulogd -Mailinglist: http://lists.gnumonks.org/mailman/listinfo/ulogd/ +Project Homepage: https://www.netfilter.org/projects/ulogd/ +Mailinglist: https://marc.info/?l=netfilter This is just a short README, pleaes see the more extensive documentation in the doc/ subdirectory. ===> IDEA